User defined pipe and redirection operators

From Rosetta Code
User defined pipe and redirection operators is a draft programming task. It is not yet considered ready to be promoted as a complete task, for reasons that should be found in its talk page.

If the language supports operator definition, then:

  • create "user defined" the equivalents of the Unix shell "<", "|", ">", "<<", ">>" and $(cmd) operators.
  • Provide simple equivalents of: cat, tee, grep, & uniq, but as filters/procedures native to the specific language.
  • Replicate the below sample shell script, but in the specific language
  • Specifically do not cache the entire stream before the subsequent filter/procedure starts. Pass each record on as soon as available through each of the filters/procedures in the chain.

Alternately: if the language does not support operator definition then replace with:

  • define the procedures: input(cmd,stream), pipe(stream,cmd), output(stream, stream), whereis(array), append(stream)
For bonus Kudos: Implement the shell "&" concept as a dyadic operator in the specific language. e.g.:
( head x & tail x & wait ) | grep test

Sample shell script: ¢ draft - pending a better (more interesting) suggestion ¢

aa="$(
  (
    head -4 < List_of_computer_scientists.lst;
    cat List_of_computer_scientists.lst | grep ALGOL | tee ALGOL_pioneers.lst;
    tail -4 List_of_computer_scientists.lst
  ) | sort | uniq | tee the_important_scientists.lst | grep aa
);
echo "Pioneer: $aa"

Input Records:

A test sample of scientists from wikipedia's "List of computer scientists"
Name Areas of interest
Wil van der Aalst business process management, process mining, Petri nets
Hal Abelson intersection of computing and teaching
Serge Abiteboul database theory
Samson Abramsky game semantics
Leonard Adleman RSA, DNA computing
Manindra Agrawal polynomial-time primality testing
Luis von Ahn human-based computation
Alfred Aho compilers book, the 'a' in AWK
Stephen R. Bourne Bourne shell, portable ALGOL 68C compiler
Kees Koster ALGOL 68
Lambert Meertens ALGOL 68, ABC (programming language)
Peter Naur BNF, ALGOL 60
Guido van Rossum Python (programming language)
Adriaan van Wijngaarden Dutch pioneer; ARRA, ALGOL
Dennis E. Wisnosky Integrated Computer-Aided Manufacturing (ICAM), IDEF
Stephen Wolfram Mathematica
William Wulf compilers
Edward Yourdon Structured Systems Analysis and Design Method
Lotfi Zadeh fuzzy logic
Arif Zaman Pseudo-random number generator
Albert Zomaya Australian pioneer of scheduling in parallel and distributed systems
Konrad Zuse German pioneer of hardware and software

These records can be declared in any format appropriate to the specific language. eg table, array, list, table or text file etc.

Output:

Pioneer: Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL

ALGOL 68

See User defined pipe and redirection operators/ALGOL 68

Works with: ALGOL 68 version Revision 1; one minor extension - PRAGMA READ; one major extension - Algol68G's Currying.
Works with: ALGOL 68G version tested with release 2.8.3.win32

File: Iterator_pipe_operators.a68

MODE
  PAGEIN =         PAGE,
  PAGEAPPEND = REF PAGE,
  PAGEOUT =    REF PAGE;

MODE
  MOID = VOID,
  YIELDLINE = PROC(LINE)VOID,
  GENLINE = PROC(YIELDLINE)VOID,
  FILTER = PROC(GENLINE)GENLINE, # the classic shell filter #
  MANYTOONE = PROC([]GENLINE)GENLINE; # eg cat, as in con[cat]enate #

PRIO =: = 5, << = 5, >> = 5;

OP <  = (FILTER filter, PAGEIN page)GENLINE: filter(READ page),
   <  = (MANYTOONE cmd, PAGEIN page)GENLINE: cmd(READ page),
   << = (FILTER filter, PAGEIN page)GENLINE: filter(READ page),
   >  = (GENLINE gen, PAGEOUT page)VOID: gen(WRITE page),
   >> = (GENLINE gen, PAGEAPPEND page)VOID: gen(APPEND page),
   =: = (GENLINE gen, FILTER filter)GENLINE: filter(gen),
   =: = (GENLINE gen, MANYTOONE cmd)GENLINE: cmd(gen);

File: Iterator_pipe_utilities.a68

 # Sample ''utilities'' PROCedure declarations #
PROC cat = ([]GENLINE argv)GENLINE:~;
PROC tee = ([]YIELDLINE args filter)FILTER:~;
PROC grep = (STRING pattern, []GENLINE argv)GENLINE:~;
PROC uniq = (GENLINE arg)GENLINE:~;
PROC sort = (GENLINE arg)GENLINE:~;
PROC head = (INT n, []GENLINE args)GENLINE:~;
PROC tail = (INT n, []GENLINE args)GENLINE:~;

File: Iterator_pipe_page.a68

# Sample ''pipe I/O'' OPerator declarations #
OP READ = (PAGEIN page)GENLINE:~;
OP WRITE = (PAGEOUT page)YIELDLINE: ~;
OP APPEND = (PAGEAPPEND page)YIELDLINE:~;

File: test_Iterator_pipe_page.a68

#!/usr/local/bin/a68g --script #
# First define what kind of record (aka LINE) we are piping and filtering #
FORMAT line fmt = $xg$;
MODE
  LINE = STRING,
  PAGE = FLEX[0]LINE,
  BOOK = FLEX[0]PAGE;

PR READ "Iterator_pipe_page.a68" PR
PR READ "Iterator_pipe_operators.a68" PR
PR READ "Iterator_pipe_utilities.a68" PR

PAGE list of computer scientists = (
  "Wil van der Aalst - business process management, process mining, Petri nets",
  "Hal Abelson - intersection of computing and teaching",
  "Serge Abiteboul - database theory",
  "Samson Abramsky - game semantics",
  "Leonard Adleman - RSA, DNA computing",
  "Manindra Agrawal - polynomial-time primality testing",
  "Luis von Ahn - human-based computation",
  "Alfred Aho - compilers book, the 'a' in AWK",
  "Stephen R. Bourne - Bourne shell, portable ALGOL 68C compiler",
  "Kees Koster - ALGOL 68",
  "Lambert Meertens - ALGOL 68, ABC (programming language)",
  "Peter Naur - BNF, ALGOL 60",
  "Guido van Rossum - Python (programming language)",
  "Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL",
  "Dennis E. Wisnosky - Integrated Computer-Aided Manufacturing (ICAM), IDEF",
  "Stephen Wolfram - Mathematica",
  "William Wulf - compilers",
  "Edward Yourdon - Structured Systems Analysis and Design Method",
  "Lotfi Zadeh - fuzzy logic",
  "Arif Zaman - Pseudo-random number generator",
  "Albert Zomaya - Australian pioneer of scheduling in parallel and distributed systems",
  "Konrad Zuse - German pioneer of hardware and software"
);

PAGE algol pioneers list, the scientists list;
PAGE aa;

# Now do a bit of plumbing: #
cat((
    head(4, ) <  list of computer scientists,
    cat(READ list of computer scientists) =: grep("ALGOL", ) =: tee(WRITE algol pioneers list),
    tail(4, READ list of computer scientists)
  )) =: sort =: uniq =: tee(WRITE the scientists list) =: grep("aa", ) >> aa;

# Finally check the result: #
print((
  "Pioneer: ", aa, newline,
  "Number of Algol pioneers: ", whole( UPB algol pioneers list, 0 ), newline, 
  "Number of scientists: ", whole( UPB the scientists list, 0 ), newline
))

Output:

Pioneer:  Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL
Number of Algol pioneers: 5
Number of scientists: 13

Go

package main

import (
    "bufio"
    "fmt"
    "io"
    "io/ioutil"
    "regexp"
    "sort"
    "strings" 
)   

// fake file system
var fs = make(map[string]string)

type file struct{ name string }

func (f file) Write(p []byte) (int, error) {
    fs[f.name] += string(p)
    return len(p), nil
}

// role of > operator
func toName(name string, src io.Reader) {
    io.Copy(file{name}, src)
}

// role of < operator
func fromName(name string) io.Reader {
    return strings.NewReader(fs[name])
}

func tee(in io.Reader, name string) io.Reader {
    return io.TeeReader(in, file{name})
}

func grep(in io.Reader, pat string) io.Reader {
    pr, pw := io.Pipe()
    go func() {
        bf := bufio.NewReader(in)
        for {
            line, readErr := bf.ReadString('\n')
            if match, _ := regexp.MatchString(pat, line); match {
                _, writeErr := io.WriteString(pw, line)
                if writeErr != nil {
                    return
                }
            }
            if readErr != nil {
                pw.CloseWithError(readErr)
                return
            }
        }
        pw.Close()
    }()
    return pr
}

func uniq(in io.Reader) io.Reader {
    pr, pw := io.Pipe()
    go func() {
        bf := bufio.NewReader(in)
        var last string
        for {
            s, readErr := bf.ReadString('\n')
            switch readErr {
            case nil:
                if s != last {
                    _, writeErr := io.WriteString(pw, s)
                    if writeErr != nil {
                        return
                    }
                    last = s
                }
                continue
            case io.EOF:
                if s > "" && s+"\n" != last { 
                    _, writeErr := io.WriteString(pw, s+"\n")
                    if writeErr != nil {
                        return
                    }
                } 
                pw.Close() 
            default:
                pw.CloseWithError(readErr)
            }
            return
        }
    }()
    return pr
}

func head(in io.Reader, lines int) io.Reader {
    pr, pw := io.Pipe()
    go func() {
        if lines <= 0 {
            lines = 10
        }
        for bf := bufio.NewReader(in); lines > 0; lines-- {
            s, readErr := bf.ReadString('\n')
            _, writeErr := io.WriteString(pw, s)
            if writeErr != nil {
                return
            }
            if readErr == nil {
                continue
            }
            if readErr == io.EOF {
                if s > "" {
                    io.WriteString(pw, "\n")
                }
            }
            pw.CloseWithError(readErr)
            return
        }
        pw.Close()
    }()
    return pr
}

func tail(in io.Reader, lines int) io.Reader {
    pr, pw := io.Pipe()
    go func() {
        if lines <= 0 {
            lines = 10
        }
        ring := make([]string, lines)
        rn := 0
        full := false
        bf := bufio.NewReader(in)
        var readErr error
        var s string
        for {
            s, readErr = bf.ReadString('\n')
            if readErr == nil {
                ring[rn] = s
                rn++
                if rn == lines {
                    rn = 0
                    full = true
                }
                continue
            }
            if readErr == io.EOF && s > "" {
                ring[rn] = s + "\n"
                rn++
            }
            break
        }
        writeLines := func(start, end int) {
            for i := start; i < end; i++ {
                if _, err := io.WriteString(pw, ring[i]); err != nil {
                    return
                }
            }
        }
        if full {
            writeLines(rn, lines)
        }
        writeLines(0, rn)
        pw.CloseWithError(readErr)
    }()
    return pr
}

func sortPipe(in io.Reader) io.Reader {
    pr, pw := io.Pipe()
    go func() {
        b, err := ioutil.ReadAll(in)
        if len(b) > 0 {
            if b[len(b)-1] == '\n' {
                b = b[:len(b)-1]
            }
            list := strings.Split(string(b), "\n")
            sort.Strings(list)
            for _, s := range list {
                if _, err := io.WriteString(pw, s+"\n"); err != nil {
                    return
                }
            }
        }
        pw.CloseWithError(err)
    }()
    return pr
}

func main() {
    fs["List_of_computer_scientists.lst"] =
        `Wil van der Aalst        business process management, process mining, Petri nets
Hal Abelson              intersection of computing and teaching
Serge Abiteboul          database theory
Samson Abramsky          game semantics
Leonard Adleman          RSA, DNA computing
Manindra Agrawal         polynomial-time primality testing
Luis von Ahn             human-based computation
Alfred Aho               compilers book, the 'a' in AWK
Stephen R. Bourne        Bourne shell, portable ALGOL 68C compiler
Kees Koster              ALGOL 68
Lambert Meertens         ALGOL 68, ABC (programming language)
Peter Naur               BNF, ALGOL 60
Guido van Rossum         Python (programming language)
Adriaan van Wijngaarden  Dutch pioneer; ARRA, ALGOL
Dennis E. Wisnosky       Integrated Computer-Aided Manufacturing (ICAM), IDEF
Stephen Wolfram          Mathematica
William Wulf             compilers
Edward Yourdon           Structured Systems Analysis and Design Method
Lotfi Zadeh              fuzzy logic
Arif Zaman               Pseudo-random number generator
Albert Zomaya            Australian pioneer of scheduling in parallel and distributed systems
Konrad Zuse              German pioneer of hardware and software
`
    toName("aa", grep(tee(uniq(sortPipe(io.MultiReader(
        head(fromName("List_of_computer_scientists.lst"), 4),
        tee(grep(fromName("List_of_computer_scientists.lst"), "ALGOL"),
            "ALGOL_pioneers.lst"),
        tail(fromName("List_of_computer_scientists.lst"), 4)))),
        "the_important_scientists.lst"), "aa"))

    fmt.Print("Pioneer: ", fs["aa"])
    showCount("Number of ALGOL pioneers", "ALGOL_pioneers.lst")
    showCount("Number of scientists", "the_important_scientists.lst")
}   

func showCount(heading, name string) {
    if data, ok := fs[name]; ok {
        lines := strings.Split(data, "\n")
        n := len(lines)
        if lines[n-1] == "" {
            n--
        }
        fmt.Printf("%s: %v\n", heading, n)
    } else {
        fmt.Println(name, "not found")
    }
}

Output:

Pioneer: Adriaan van Wijngaarden  Dutch pioneer; ARRA, ALGOL
Number of ALGOL pioneers: 5
Number of scientists: 13

J

If we ignore the gratuitous complexity requirements of this task, it boils down to this:

Step 0: get the data. The task does not specify how to get the data, so here I use lynx, which is readily available on most unix-like systems, including cygwin. Note that lynx needs to be in the OS PATH when running j.

require 'task'
data=:<;._2 shell 'lynx -dump -nolist -width=999 http://en.wikipedia.org/wiki/List_of_computer_scientists'

Step 1: define task core algorithms:

grep=: +./@E.S:0 # ]

Step 2: select and display the required data:

   ;'aa' grep 'ALGOL' grep data
     * Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL

As for the concept of a pipe that presents data one record at a time to a downstream function, that corresponds to the J operator @ and we could achieve the "left to right" syntax mechanism by explicitly ordering its arguments 2 :'v@u' but it's not clear how to demonstrate that usefully, in this task. (And, I could write a lot of code, to accomplish what's being accomplished here with the two successive greps, but I find that concept distasteful and tedious.)

However, note also that J's sort (/:~) and uniq (~.) operations would work just fine on this kind of data. For example:

   ;'aa' grep 'ALGOL' grep data,data
     * Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL
     * Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL

   ;'aa' grep ~. 'ALGOL' grep data,data
     * Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL

That said, this implements most (perhaps all) of the required complexities:

declare=: erase@boxopen
tee=: 4 :0
  if._1=nc boxopen x do.(x)=: '' end.
  (x)=: (do x),y
  y
)
grep=: 4 :'x (+./@E.S:0 # ]) y'
pipe=:2 :'v@(u"0)'  NB. small pipe -- spoon feed one record at a time
PIPE=:2 :0          NB. big pipe   -- feed everything all together
  v u y
:
  v (,x)"_ y        NB. syntactic sugar, beware of tooth decay
) 
head=: {.
tail=: -@[ {. ]
sort=: /:~
uniq=: ~.
cat=:  ]
echo=: smoutput@;

declare;:'ALGOL_pioneers the_important_scientists'
aa=: ;do TXT=:0 :0 -.LF
  (
    (
      4 head data
    ),(
      cat pipe
      ('ALGOL'&grep) pipe
      ('ALGOL_pioneers'&tee)
        data
    ),(
      4 tail data
  )) ''PIPE
  sort PIPE
  uniq PIPE
  ('the_important_scientists'&tee) PIPE
  ('aa'&grep)
    ''
)

echo 'Pioneer:';aa

This produces the result:

Pioneer:     * Adriaan van Wijngaarden - Dutch pioneer; ARRA, ALGOL

Julia

Uses Julia's Channels as asynchronous pipes. Note that the $( .. ) Unix command, which gathers stdin output into a single variable, is not used. Instead, a common output channel for various tasks is used to gather output. Sort() rather than sort() is used so as not to overload Base.sort().

const LISTDATA = """
Wil van der Aalst	business process management, process mining, Petri nets
Hal Abelson	intersection of computing and teaching
Serge Abiteboul	database theory
Samson Abramsky	game semantics
Leonard Adleman	RSA, DNA computing
Manindra Agrawal	polynomial-time primality testing
Luis von Ahn	human-based computation
Alfred Aho	compilers book, the 'a' in AWK
Stephen R. Bourne	Bourne shell, portable ALGOL 68C compiler
Kees Koster	ALGOL 68
Lambert Meertens	ALGOL 68, ABC (programming language)
Peter Naur	BNF, ALGOL 60
Guido van Rossum	Python (programming language)
Adriaan van Wijngaarden	Dutch pioneer; ARRA, ALGOL
Dennis E. Wisnosky	Integrated Computer-Aided Manufacturing (ICAM), IDEF
Stephen Wolfram	Mathematica
William Wulf	compilers
Edward Yourdon	Structured Systems Analysis and Design Method
Lotfi Zadeh	fuzzy logic
Arif Zaman	Pseudo-random number generator
Albert Zomaya	Australian pioneer of scheduling in parallel and distributed systems
Konrad Zuse	German pioneer of hardware and software
"""

datafilename = "List_of_computer_scientists.lst"
stat(datafilename).size == 0 && (fd = open(datafilename, "a"); write(fd, LISTDATA); close(fd))

channelstream() = Channel{String}(200)
closewhenempty(c) = @async begin while !isempty(c) sleep(rand()) end; close(c); end

function head(filename, numlines, chan=channelstream())
    fd = open(filename)
    buffer = String[]
    while !eof(fd)
        push!(buffer, readline(fd, keep=true))
        length(buffer) == numlines && break
    end
    close(fd)
    @async begin
        for line in buffer
            put!(chan, line)
        end
        closewhenempty(chan)
    end
    return chan
end

function cat(filename::AbstractString, chan=channelstream())
    @async begin
        fd = open(filename)
        while !eof(fd)
            put!(chan, readline(fd, keep=true))
        end
        close(fd)
        closewhenempty(chan)
    end
    return chan
end

function grep(inchan::Channel, target, outchan = channelstream())
    @async begin
        try
            while isopen(inchan)
                line = take!(inchan)
                if occursin(target, line)
                    put!(outchan, line)
                end
            end
        catch;
        end
        closewhenempty(outchan)
    end
    return outchan
end
grep(target) = (chan) -> grep(chan, target)

function tee(inchan::Channel, filename, outchan=channelstream())
    fd = open(filename, "w")
    @async begin
        while isopen(inchan)
            try
                line = take!(inchan)
                write(fd, line)
                put!(outchan, line)
            catch;
                break
            end
        end
        close(fd)
        closewhenempty(outchan)
    end
    return outchan
end
tee(filename, outchan=channelstream()) = (inchan) -> tee(inchan, filename, outchan)

function tail(filename, numlines, chan=channelstream())
    fd = open(filename)
    buffer = String[]
    while !eof(fd)
        push!(buffer, readline(fd, keep=true))
        length(buffer) > numlines && popfirst!(buffer)
    end
    @async begin
        for line in buffer
            put!(chan, line)
        end
        closewhenempty(chan)
    end
    return chan
end

function Sort(inchan, outchan = channelstream())
    buffer = String[]
    try
        while isopen(inchan)
            push!(buffer, take!(inchan))
        end
    catch;
    end
    @async begin
        for line in sort!(buffer)
            put!(outchan, line)
        end
        closewhenempty(outchan)
    end
    return outchan
end
Sort() = (chan) -> Sort(chan)

function uniq(inchan, outchan = channelstream())
    alreadyseen = Set{String}()
    @async begin
        while isopen(inchan)
            try
                line = take!(inchan)
                if !(line in alreadyseen)
                    push!(alreadyseen, line)
                    put!(outchan, line)
                end
            catch;
                break
            end
        end
        closewhenempty(outchan)
    end
    return outchan
end
uniq() = (chan) -> uniq(chan)

function print_lines(chan)
    @async begin
        while isopen(chan)
            line = take!(chan)
            print(line)
        end
    end
end
print_lines() = (chan) -> print_lines(chan)

const commonoutchan = channelstream()

head("List_of_computer_scientists.lst", 4, commonoutchan)

cat("List_of_computer_scientists.lst") |> grep("ALGOL") |> tee("Algol_pioneers.lst", commonoutchan)

tail("List_of_computer_scientists.lst", 4, commonoutchan)

Sort(commonoutchan) |> uniq() |> tee("the_important_scientists.lst") |>
    grep("aa") |> print_lines()
Output:
Adriaan van Wijngaarden Dutch pioneer; ARRA, ALGOL

Perl

Implementing only stream chaining, cat, grep and tee. Oddly enough, I don't feel the urge to implement all of the more-or-less-the-same features asked for by the task.

use strict;
use 5.10.0;

package IO::File;
sub readline { CORE::readline(shift) } # icing, not essential

package Stream;
use Exporter 'import';

# Only overload one operator.  "file | stream" and "stream | stream"
# are not ambiguous like with shell commands.
use overload '|' => \&chain;
sub new {
	my $cls = shift;
	bless { args => [@_] }, ref $cls || $cls;
}

sub chain {
	my ($left, $right, $swap) = @_;
	($left, $right) = ($right, $left) if $swap;

	if (!ref $left) {
		my $h;
		open $h, $left and $left = $h or die $left
	}

	if (!ref $right) {
	# output file not implemented: don't know where I'd ever use it
		my $h;
		open $h, '>', $right	and $right = $h or die $right
	}

	if (ref $left and $left->isa(__PACKAGE__)) {
		$left->{output} = $right;
	}

	if (ref $right and $right->isa(__PACKAGE__)) {
		$right->{input} = $left;
	}
	$right;
}

# Read a line and do something to it.  By default it's this dummy
# pass-through function.  Overriding it defines a subclass' behavior
sub transform { shift; shift }

sub readline {
	my $obj = shift;
	my $line;
	return $line = <STDIN> unless defined $obj->{input};

	while (1) {
		$line = $obj->{input}->readline	or return;
		return $line if $line = $obj->transform($line);
	}
}

package Cat;
use parent -norequire, 'Stream';
# Dummy, exactly the same as Stream.  Except now we can invoke
# as Cat::ter, instead of Stream::ter, which is not even a word
sub ter { Cat->new(@_) }

package Grep;
use parent -norequire, 'Stream';

sub transform {
	my ($obj, $line) = @_;
	for (@{$obj->{args}}) {
		return $line if ($line =~ $_)
	}
	return;
}

sub per { Grep->new(@_) }

package Tee;
use parent -norequire, 'Stream';
sub er{
	my $obj = Tee->new(@_);
	@{$obj->{tees}} =
		map { open my $h, '>', $_ or die $_; $h }
		@{$obj->{args}};
	delete $obj->{args};
	$obj
}

sub transform {
	my ($obj, $line) = @_;
	print $_ $line for @{$obj->{tees}};
	$line;
}

package main;
my $chain =
	'/etc/services'		# head of chain; omit to use STDIN
	| Cat::ter		# don't really need this line
	| Grep::per(qr/tcp/)
	| Tee::er('/tmp/t1', '/tmp/t2')
	| Grep::per(qr/170/)
	| Tee::er('/tmp/t3')
	;

print while $_ = $chain->readline;

Phix

Translation of: Go
Library: Phix/Class

You could of course do things more character-by-character or line-by-line, and/or farm things out to separate threads/tasks, but the latter would need some suspend/resume/scheduling, along with explicit eof markers. The distributed version also has a couple of alternatives for pipe_head() and pipe_tail(), along with a class-less version that is compatible with pwa/p2js.

-- demo\rosetta\Fake_Redirection.exw
without js -- class (see other version in distro)

constant fs = new_dict()    -- fake file system

class pipe                  -- fake pipe
  string data = ""
  integer idx = 0
  function getch()
    if idx<length(data) then
        idx += 1
        return data[idx] 
    end if
    return -1
  end function
  function getln()
    if idx<length(data) then
        integer start = idx+1
        idx = find('\n',data,start)
        return data[start..idx]
    end if
    return -1
  end function
  procedure putch(integer ch)
    data &= ch
  end procedure
  procedure putln(string line)
    data &= line
  end procedure
  function readall()
    return split(data,'\n',no_empty:=true)
  end function
  function rawdata()
    return data
  end function
end class

function joinup(sequence lines)
    return join(lines,"\n")&"\n"
end function

procedure toName(string name, pipe src) -- role of > operator
    setd(name,src.rawdata(),fs)
end procedure
 
function fromName(string name)          -- role of < operator
    return new(pipe,{getd(name,fs)})
end function
 
function tee(pipe pin, string name)
    string data = pin.rawdata()
    setd(name,data,fs)
    return new(pipe,{data})
end function
 
function grep(pipe pin, string pat)
    pipe res = new()
    while true do
        object line = pin.getln()
        if atom(line) then exit end if
        if match(pat,line) then
            res.putln(line)
        end if
    end while
    return res
end function

function multireader(sequence pipes)
    pipe res = new()
    for i=1 to length(pipes) do
        pipe p = pipes[i]
        res.putln(p.rawdata())
    end for
    return res
end function

function pipe_head(pipe pin, integer lines)
    pipe res = new()
    for i=1 to lines do
        object line = pin.getln()
        if atom(line) then exit end if
        res.putln(line)
    end for
    return res
-- or, nicer/neater but potentially much wasted effort:
--  return new(pipe,{joinup(head(pin.readall(),lines))})
end function

function pipe_tail(pipe pin, integer lines)
--  sequence ring = repeat("",lines)
--  integer rn = 0, full = false
--  while true do
--      object line = pin.getln()
--      if atom(line) then exit end if
--      rn += 1
--      if rn>lines then {rn,full} = {1,true} end if
--      ring[rn] = line
--  end while
--  ring = iff(full?ring[rn+1..$]:{}) & ring[1..rn]
--  return new(pipe,{joinup(ring)})
    return new(pipe,{joinup(tail(pin.readall(),lines))})
end function

function sort_unique(pipe pin)
    return new(pipe,{joinup(unique(pin.readall()))})
end function
 
procedure showCount(string heading, name)
    if getd_index(name,fs)=NULL then crash("not found") end if
    integer n = length(split(getd(name,fs), "\n"))
    printf(1,"%s: %d\n", {heading, n})
end procedure

constant lcs_txt = """
Wil van der Aalst        business process management, process mining, Petri nets
Hal Abelson              intersection of computing and teaching
Serge Abiteboul          database theory
Samson Abramsky          game semantics
Leonard Adleman          RSA, DNA computing
Manindra Agrawal         polynomial-time primality testing
Luis von Ahn             human-based computation
Alfred Aho               compilers book, the 'a' in AWK
Stephen R. Bourne        Bourne shell, portable ALGOL 68C compiler
Kees Koster              ALGOL 68
Lambert Meertens         ALGOL 68, ABC (programming language)
Peter Naur               BNF, ALGOL 60
Guido van Rossum         Python (programming language)
Adriaan van Wijngaarden  Dutch pioneer; ARRA, ALGOL
Dennis E. Wisnosky       Integrated Computer-Aided Manufacturing (ICAM), IDEF
Stephen Wolfram          Mathematica
William Wulf             compilers
Edward Yourdon           Structured Systems Analysis and Design Method
Lotfi Zadeh              fuzzy logic
Arif Zaman               Pseudo-random number generator
Albert Zomaya            Australian pioneer of scheduling in parallel and distributed systems
Konrad Zuse              German pioneer of hardware and software
""",
mainlist = "List_of_computer_scientists.lst"
setd(mainlist,lcs_txt,fs)
toName("aa", grep(tee(sort_unique(multireader({pipe_head(fromName(mainlist), 4),
                                               tee(grep(fromName(mainlist), "ALGOL"),
                                                   "ALGOL_pioneers.lst"),
                                               pipe_tail(fromName(mainlist), 4)})),
                      "the_important_scientists.lst"),
                  "aa"))
 
printf(1,"Pioneer: %s", getd("aa",fs))
showCount("Number of ALGOL pioneers", "ALGOL_pioneers.lst")
showCount("Number of scientists", "the_important_scientists.lst")

?"done"
{} = wait_key()
abort(0)
Output:
Pioneer: Adriaan van Wijngaarden  Dutch pioneer; ARRA, ALGOL
Number of ALGOL pioneers: 5
Number of scientists: 13

Racket

#lang racket

(module racksh racket

  (require (for-syntax racket/syntax))

  (provide (except-out (all-from-out racket) #%app sort)
           (rename-out [shell-app #%app]))

  (define-syntax (shell-app stx)
    (define (=? x y)
      (eq? (if (syntax? x) (syntax-e x) x) (if (syntax? y) (syntax-e y) y)))
    (define (err msg) (raise-syntax-error 'shell msg stx))
    (define (make-call xs)
      (cond [(null? xs) (err "empty form")]
            [(string? (syntax-e (car xs))) #`(shell #,(car xs) #,@(cdr xs))]
            [(eq? #\{ (syntax-property (car xs) 'paren-shape)) (cons #'void xs)]
            [else xs]))
    (syntax-case stx ()
      [(_ x ...) (eq? #\{ (syntax-property stx 'paren-shape))
       (let loop ([xs (reverse (syntax->list #'(\; x ...)))]
                  [form '()] [thunks '()] [I #f] [O #f] [seq '()])
         (cond [(null? xs) #`(begin #,@seq)]
               [(=? '\; (car xs))
                (loop (cdr xs) '() '() #f #f
                      (if (and (null? form) (null? thunks)) seq
                          (let* ([form (make-call form)]
                                 [r (if (null? thunks) form
                                        #`(pipe (list (λ() #,form) #,@thunks)))]
                                 [r (if I #`(#,@I (λ() #,r)) r)]
                                 [r (if O #`(#,@O (λ() #,r)) r)])
                            (cons r seq))))]
               [(=? '\| (car xs))
                (loop (cdr xs) '() (cons #`(λ() #,(make-call form)) thunks) I O
                      seq)]
               [(or (=? '< (car xs)) (=? '<< (car xs)))
                (cond [(null? form) (err "missing expression after < or <<")]
                      [I (err "duplicate < or << specified")]
                      [else (loop (cdr xs) (cdr form) thunks
                                  (if (=? '< (car xs))
                                    #`(with-input-from-file #,(car form))
                                    #`(with-input-from-string #,(car form)))
                                  O seq)])]
               [(or (=? '> (car xs)) (=? '>> (car xs)))
                (cond [(null? form) (err "missing expression after > or >>")]
                      [O (err "duplicate > or >> specified")]
                      [else (loop (cdr xs) (cdr form) thunks I
                                  #`(with-output-to-file #,(cadr xs) #:exists
                                      '#,(if (=? '> (car xs)) 'truncate 'append))
                                  seq)])]
               [else (loop (cdr xs) (cons (car xs) form) thunks I O seq)]))]
      [(_ x ...) #'(x ...)]))

  (define (pipe thunks)
    (if (null? (cdr thunks)) ((car thunks))
        (let-values ([(I O) (make-pipe)])
          (parameterize ([current-output-port O])
            (thread (λ() (dynamic-wind void (car thunks)
                                       (λ() (close-output-port O))))))
          (parameterize ([current-input-port I]) (pipe (cdr thunks))))))

  (define (shell cmd . args)
    (apply system* (find-executable-path cmd)
           (map (λ(x) (if (string? x) x
                          (with-output-to-string (λ() (display x)))))
                args)))

  ;; implements a common interface of reading a bunch of files; '- means
  ;; stdin; no files means just stdin
  (define (call/files files proc)
    (if (null? files) (proc (current-input-port))
        (let-values ([(I O) (make-pipe)])
          (thread
           (λ() (for ([file (in-list files)])
                  (if (eq? '- file)
                    (copy-port (current-input-port) O)
                    (call-with-input-file file (λ(i) (copy-port i O)))))
                (close-output-port O)))
          (proc I))))

  (define-syntax (define-io stx)
    (syntax-case stx ()
      [(_ (name . xs) E ...)
       (with-syntax ([io-name (format-id #'name "io-~a" #'name)])
         #'(begin (provide (rename-out [io-name name]))
                  (define (io-name . xs) E ...)))]))

  (define-io (echo . xs)
    (for-each display (add-between xs " "))
    (newline))
  (define-io (cat . files)
    (call/files files (λ(I) (copy-port I (current-output-port)))))
  (define-io (sort . files)
    (display-lines (sort (call/files files port->lines) string<?)))
  (define-io (head n . files)
    (call/files files
      (λ(I) (for ([l (in-lines I)] [i (in-range n)]) (displayln l)))))
  (define-io (tail n . files)
    (display-lines (take-right (call/files files port->lines) n)))
  (define-io (grep rx . files)
    (call/files files
      (λ(I) (for ([l (in-lines I)] #:when (regexp-match? rx l)) (displayln l)))))
  (define-io (uniq . files)
    (call/files files
     (λ(I) (let loop ([last #f])
             (define line (read-line I))
             (unless (eof-object? line)
               (unless (equal? line last) (displayln line))
               (loop line))))))
  (define-io (tee file)
    (call-with-output-file file #:exists 'truncate
      (λ(O) (for ([l (in-lines (current-input-port))])
              (displayln l O) (displayln l)))))

  (provide $)
  (define-syntax-rule ($ E ...)
    (with-output-to-string (λ() E ...))))

(module sample (submod ".." racksh)
  {\;
   define file "List_of_computer_scientists.lst" \;
   define aa ($
     {{ head 4 < file \;
        cat file \| grep "ALGOL" \;
        tail 4 < file \;
      } \| sort \| uniq \| tee "the_important_scientists.lst" \| grep "aa"
     }) \;
   echo "Pioneer:" aa}
  )

(require 'sample)

Raku

(formerly Perl 6) Implementing cat, redirect(<), head, tail and tee. I share the same general lack of enthusiasm as the Perl example author to implement the other shell ops. Many of the ops already exist in some form in Raku, though they may have slightly different syntax and precedence as the shell ops. I didn't bother implementing pipe (|) as Raku pretty much has chaining semantics built in.

The less than '<' operator is very low level in Raku and would be troublesome to override, so is implemented as 'redirect'. The sort and unique directives don't really have any effect on the final result string, it would be the same without them, but it is following the task example.

This assumes that the data is in a tab separated file "List_of_computer_scientists.lst" in the current directory.

sub cat ($fname) { lazy $fname.IO.lines };
sub head ($count, @positional) { @positional.head($count) }
sub redirect ($fname) { cat($fname) }
sub tail (Int $count, Str $fname) { $fname.IO.lines.tail($count) }
sub infix:<tee> ($stream, $fname) {
    # need to reify the lazy list to write to a file
    my @values = @$stream[^Inf].grep: *.defined;
    # meh. not really going to write the files
    # $fname.IO.put @values.join("\n")
    # just pass the reified values along
    @values.List
}

my $aa = ((
    (head 4, redirect 'List_of_computer_scientists.lst'),
    cat('List_of_computer_scientists.lst') .grep( /'ALGOL'/ ) tee 'ALGOL_pioneers.lst',
    tail 4, 'List_of_computer_scientists.lst'
).flat .sort .unique tee 'the_important_scientists.lst') .grep: /'aa'/;

say "Pioneer: $aa";
Output:
Pioneer: Adriaan van Wijngaarden	Dutch pioneer; ARRA, ALGOL

Tcl

The syntax of redirections is slightly out, as they're inserted as explicit pipeline elements, and standard Tcl syntax is used to pull in results from sub-pipelines (because it is vastly simpler):

package require Tcl 8.6

# Helpers
proc aspipe {input cmd args} {
    tailcall coroutine pipe[incr ::pipes] eval {yield [info coroutine];} \
	[list $cmd $input {*}$args] {;break}
}
proc forpipe {input var body} {
    upvar 1 $var v
    while {[llength [info commands $input]]} {
	set v [$input]
	uplevel 1 $body
    }
}

# Pipeline framework; parses, collects results as newline-separated lines
proc pipeline args {
    if {![llength $args]} {error "no pipeline components"}
    set p [aspipe {} eval {while {[gets stdin line]>=0} {yield $line}}]
    set oi -1
    foreach ni [lsearch -all [lappend args "|"] "|"] {
	set cmd [lrange $args [expr {$oi+1}] [expr {$ni-1}]]
	set p [aspipe $p {*}$cmd]
	set oi $ni
    }
    set accum {}
    forpipe $p line {
	lappend accum $line
    }
    return [join $accum \n]
}

# Pipeline implementations - redirections
proc << {in args} {
    foreach string $args {
	foreach line [split $string "\n"] {
	    yield $line
	}
    }
}
proc < {in filename} {
    set f [open $filename]
    while {[gets $f line] >= 0} {
	yield $line
    }
    close $f
}
proc > {in filename} {
    set f [open $filename w]
    forpipe $in line {
	puts $f $line
    }
    close $f
}
proc >> {in filename} {
    set f [open $filename a]
    forpipe $in line {
	puts $f $line
    }
    close $f
}

# Pipeline implementations - "commands"
proc cat {in args} {
    foreach filename $args {
	if {$filename eq "-"} {
	    forpipe $in line {
		yield $line
	    }
	} else {
	    set f [open $filename]
	    while {[gets $f line] >= 0} {
		yield $line
	    }
	    close $f
	}
    }
}
proc head {in count} {
    forpipe $in line {
	if {[incr i] <= $count} {
	    yield $line
	}
    }
}
proc tail {in count} {
    incr count -1
    set accum {}
    forpipe $in line {
	set accum [lrange [lappend accum $line] end-$count end]
    }
    foreach item $accum {yield $item}
}
proc grep {in RE} {
    forpipe $in line {
	if {[regexp $RE $line]} {yield $line}
    }
}
proc sort {in} {
    set accum {}
    forpipe $in line {
	lappend accum $line
    }
    foreach line [lsort $accum] {yield $line}
}
proc uniq {in} {
    forpipe $in line {
	if {![info exists prev] || $prev ne $line} {
	    yield $line
	}
	set prev $line
    }
}
proc wc {in {type "words"}} {
    set count 0
    switch $type {
	words { set RE {\S+} }
	lines { set RE {.*}  }
    }
    forpipe $in line {
	incr count [regexp -all $RE $line]
    }
    yield $count
}
proc tee {in filename} {
    set f [open $filename w]
    forpipe $in line {
	puts $f $line
	yield $line
    }
    close $f
}

Sample pipeline:

set file "List_of_computer_scientists.lst"
set aa [pipeline \
    << [pipeline < $file | head 4] [pipeline < $file | grep ALGOL | tee "ALGOL_pioneers.txt"] [pipeline < $file | tail 4] \
    | sort | uniq | tee "the_important_scientists.lst" | grep aa]
puts "Pioneer: $aa"

Wren

Translation of: Phix
Library: Wren-seq

Although Wren supports operator overloading, there are a number of restrictions which would make simulating the Unix shell operators awkward or even imposible. As in the Phix (and Go) examples, I've therefore used named methods instead.

import "./seq" for Lst

var FS = {}   // fake file system

var JoinUp = Fn.new { |lines| lines.join("\n") + "\n" }

class Pipe {  // fake pipe 
    static fromName(name) { Pipe.new(FS[name]) }  // role of < operator

    static multireader(pipes) {
        var res = Pipe.new("")
        for (i in 0...pipes.count) {
            var p = pipes[i]
            res.putln(p.rawData)
        }
        return res
    }

    construct new(data) {
        _data = data
        _idx = -1
    }

    getln() {
        if (_idx < _data.count-1) {
            var start = _idx + 1
            _idx = _data.indexOf("\n", start)
            return (_idx >= 0) ? _data[start.._idx] : ""
        }
        return ""
    }

    putln(line) { _data = _data + line }

    readAll() { _data.split("\n").where { |s| s != "" }.toList }

    rawData { _data }

    toName(name) { FS[name] = _data }   // role of > operator

    tee(name) { Pipe.new(FS[name] = _data) } 

    grep(pat) {
        var res = Pipe.new("")
        while (true) {
            var line = getln()
            if (line == "") break
            if (line.indexOf(pat) >= 0) res.putln(line)
        }
        return res
    }

    head(lines) { Pipe.new(JoinUp.call(readAll().take(lines).toList)) }

    tail(lines) {
        var t = readAll()
        if (t.count >= lines) t = t[-lines..-1]
        return Pipe.new(JoinUp.call(t))
    }

    sortUnique { Pipe.new(JoinUp.call(Lst.distinct(readAll()))) }
}

var showCount = Fn.new { |heading, name|
    if (!FS[name]) Fiber.abort("not found")
    var n = FS[name].split("\n").count { |s| s != "" }
    System.print("%(heading): %(n)")
}

var lcsTxt = """
Wil van der Aalst        business process management, process mining, Petri nets
Hal Abelson              intersection of computing and teaching
Serge Abiteboul          database theory
Samson Abramsky          game semantics
Leonard Adleman          RSA, DNA computing
Manindra Agrawal         polynomial-time primality testing
Luis von Ahn             human-based computation
Alfred Aho               compilers book, the 'a' in AWK
Stephen R. Bourne        Bourne shell, portable ALGOL 68C compiler
Kees Koster              ALGOL 68
Lambert Meertens         ALGOL 68, ABC (programming language)
Peter Naur               BNF, ALGOL 60
Guido van Rossum         Python (programming language)
Adriaan van Wijngaarden  Dutch pioneer; ARRA, ALGOL
Dennis E. Wisnosky       Integrated Computer-Aided Manufacturing (ICAM), IDEF
Stephen Wolfram          Mathematica
William Wulf             compilers
Edward Yourdon           Structured Systems Analysis and Design Method
Lotfi Zadeh              fuzzy logic
Arif Zaman               Pseudo-random number generator
Albert Zomaya            Australian pioneer of scheduling in parallel and distributed systems
Konrad Zuse              German pioneer of hardware and software
"""

var mainList = "List_of_computer_scientists.lst"
FS[mainList] = lcsTxt
var p = Pipe.fromName(mainList)
var pipes = [p.head(4), p.grep("ALGOL").tee("ALGOL_pioneers.lst"), p.tail(4)]
var p2 = Pipe.multireader(pipes).sortUnique.tee("the_important_scientists.lst").grep("aa").toName("aa")
System.write("Pioneer: %(FS["aa"])")
showCount.call("Number of ALGOL pioneers", "ALGOL_pioneers.lst")
showCount.call("Number of scientists", "the_important_scientists.lst")
Output:
Pioneer: Adriaan van Wijngaarden  Dutch pioneer; ARRA, ALGOL
Number of ALGOL pioneers: 5
Number of scientists: 13