Proposal: Hack-style Pipelining

[zenparsing]

The current proposal (in which the RHS is implicitly called with the result of the LHS) does not easily support the following features:

• Piping into a function with multiple arguments
• Calling methods on a piped value
• Awaiting a piped value

In order to better support these features the current proposal introduces special-case syntax and requires the profilgate use of single-argument arrow functions within the pipe.

This proposal modifies the semantics of the pipeline operator so that the RHS is not implicitly called. Instead, a constant lexical binding is created for the LHS and then supplied to the RHS. This is similar to the semantics of Hack's pipe operator.

Runtime Semantics

PipelineExpression : PipelineExpression |> LogicalORExpression

1. Let left be the result of evaluating PipelineExpression.
2. Let leftValue be ? GetValue(left).
3. Let oldEnv be the running execution context's LexicalEnvironment.
4. Let pipeEnv be NewDeclarativeEnvironment(oldEnv).
5. Let pipeEnvRec be pipeEnv's EnvironmentRecord.
6. Perform ! pipeEnvRec.CreateImmutableBinding("$", true).
7. Perform ! pipeEnvRec.InitializeBinding("$", leftValue);
8. Set the running execution context's LexicalEnvironment to pipeEnv.
9. Let right be the result of evaluating LogicalORExpression.
10. Set the running execution context's LexicalEnvironment to oldEnv.
11. Return right.

Example

anArray
  |> pickEveryN($, 2)
  |> $.filter(...)
  |> shuffle($)
  |> $.map(...)
  |> Promise.all($)
  |> await $
  |> $.forEach(console.log);

Advantages

• All use cases described above are supported with no special-case syntax.
• By removing the implicit-call behavior, we remove a potential source of confusion for developers.

Disadvantages

• By requiring an explicit function call, functional programming patterns that rely heavily on currying will be less ergonomic.

Notes

The choice of "$" for the lexical binding name is somewhat arbitrary: it could be any identifier. It should probably be one character and should ideally stand out from other variable names. For these reasons, "$" seems ideal. However, this might result in a conflict for users that want to combine both jQuery and the pipeline operator. Personally, I think it would be a good idea to discourage usage of "$" and "_" as variable names with global meanings. We have modules; we don't need jQuery to be "$" anymore!

[littledan]

Thanks for writing this out in such a detailed way. I didn't understand exactly what you were suggesting before. I like how this direction is very explicit and general; it completely handles nested subexpressions in a way that I didn't previously understand was possible.

The cost seems to be the loss of terse syntax for x |> f |> g, which becomes x |> f($) |> g($). I'm not sure how important conciseness for that case is.

Bikeshedding: I'm not sure if we want to use _ or $ when they are so widely used for libraries. There are lots of other names possible, or we could go with something currently untypable like <>.

[zenparsing]

I'm not sure if we want to use _ or $ when they are so widely used for libraries.

True, using $ or _ will probably make some people unhappy. On the other hand, we should prioritize the future over the past.

Another thought: a pipe operator with these semantics would obviate the need for the current :: proposal. We could re-purpose binary :: for just method extraction:

class XElem extends HTMLElement {
  _onClick() {
    // ...
  }

  connectedCallback() {
    this.addEventListener('click', this::_onClick);
  }

  disconnectedCallback() {
    // Assuming this::_onClick === this::_onClick
    this.removeEventListener('click', this::_onClick);
  }
}

which would be pretty nice and intuitive.

[gilbert]

Nicely done! The loss of terse syntax for unary and curried functions is arguably significant, but in return you get pretty much everything else one could ask for 😄

Some more fun features of this syntax:

let lastItem =
  getArray() |> $[$.length-1]

let neighbors =
  array.indexOf('abc') |> array.slice($ - 1, $ + 2)

let silly =
  f |> $(x, y)

[dead-claudia]

This looks a lot like a variant of the partial application proposal, just using a $ (a valid identifier) instead of a ? (which is what that proposal uses).

Maybe we could take some of the features proposed here and integrate them into that proposal, but from the surface, this looks flat out identical to that proposal mod the change in syntactic marker.

[mAAdhaTTah]

I guess I'm surprised $ would even be possible. Something like the $.map example would be especially confusing because jQuery has a $.map. Could the grammar specify whether that's supposed to call immediately or not? If someone did this:

const $ = require('ramda') // using ramda cuz I'm familiar

[1, 2] |> $.map(x => x + 1)

it can't tell when it should use the return value of Ramda's curried map or the Array.prototype.map. Is using a valid variable name explicitly intended?

@isiahmeadows My understanding is this proposal explicitly evolved out of prior attempts to combine the two proposals.

Given that, though is the intention to actually combine the proposals or use the placeholder semantics defined as part of the pipeline proposal as a springboard for adding partial application to the language generally?

[js-choi]

I'm not actually proposing this, but, for what it's worth, the Clojure programming language has a similar pipelining feature called the as-> macro. It has a somewhat different approach, though: it requires explicitly specifying the placeholder argument at the beginning of the pipeline—which may be any valid identifier.

That is, the example in the original post would become something like:

anArray |> ($) { // The $ may be any arbitrary identifier
  pickEveryN($, 2),
  $.filter(...),
  shuffle($),
  $.map(...),
  Promise.all($),
  await $,
  $.forEach(console.log)
}

…but the $ could instead be _ or x or ス or any other valid identifier. That's the advantage: It doesn't implicitly clobber a specially privileged identifier; it requires an explicit identifier for its placeholder argument binding. The downside is that it needs to have its own block—which would contain a list of expressions through which to pipe placeholder data, and within which the placeholder identifier would resolve—rather than the current |> binary operator. That's probably not tenable, I'm guessing.

Having said all that, like @isiahmeadows I too am wondering if ?, as a nonidentifier, would not be a better choice here. The relationship between this proposal and the partial-application proposal may need to be once again considered.

[JAForbes]

I see the benefit of this for multi arg functions, but I don't understand why we need a placeholder for single arg functions? A lot of the FP community in JS has moved or is moving to unary only functions, and so for us it'd be quite natural to do x |> f |> g |> h, littering that expression with $ (or some other symbol) for every application would be pretty noisy, so I'd like to know the benefit.

[JAForbes]

I'm also really surprised await is even a part of this proposal.

anArray
  |> pickEveryN($, 2)
  |> $.filter(...)
  |> shuffle($)
  |> $.map(...)
  |> Promise.all($)
  |> await $
  |> $.forEach(console.log);

Should just be with lodash/fp, ramda, sanctuary ( etc )

anArray
  |> pickEveryN(2)
  |> filter(f)
  |> shuffle
  |> map(f)
  |> Promise.all
  |> then( forEach(console.log) )

|> is a very simple operator, it just applies a function with the result of the previous value, it doesn't need any other semantics.

If we want to use legacy OO style libraries with this syntax, we should have a separate proposal (like the placeholder proposal), and it should not be pipeline specific, it should be language wide. The worst case scenario with arrow functions is completely fine. And the fact it's not as convenient as unary functions is a good thing, it's a subtle encouragement to think compositionally and design API's that enable that.

E.g. let's say we're using pipeline with no placeholder, with no await, with standard lodash and bluebird.

anArray
  |> x => pickEveryN(x, 2)
  |> x => filter(x, f)
  |> shuffle
  |> x => map(x, f)
  |> Promise.all
  |> then( xs => xs.forEach(console.log) )

It's not bad for an API that is not designed for unary application.

I think we may be over-complicating things to support functionality we don't need to support. The work-around of using arrow functions is still simple to follow, and any special placeholder syntax should be language wide anyway and probably beyond the scope of this proposal.

[mAAdhaTTah]

Unfortunately, latest guidance from TC39 is they want await support. This is a syntax that will need to be solved before it can advance.

[TehShrike]

Is nobody at TC39 willing to oppose await in this proposal? It seems like a bad idea to take this super-handy operator common to other languages and bolt on this awkward feature.

[TehShrike]

To be fair, I haven't heard the discussion, so maybe I'm missing something?

[TehShrike]

The notes from the discussion seem more negative towards the idea of bolting on await/yield than anything http://tc39.github.io/tc39-notes/2017-11_nov-29.html#interaction-of-await-and-pipeline

[JAForbes]

Yeah I agree, and the arguments for await are either not justifications or they are reaching for exotic scenarios.

KCL: There are valid usecases for await expressions in a pipeline, an async function that returns a function could be used here.

This is supported by the above |> then( f => f(x) ). Also if exotic examples are required to justify this syntax, then it's not really justifiable.

DE: Yes, I think it'd be very strange if we didn't allow this.

I think we need more guidance than this.

Its a very simple operator, we don't need to complicate it. I think @zenparsing's suggestion is an elegant solution for the constraints tc39 have provided, but I don't think the constraints have any merit and we should seek further guidance.

If we ship it with await banned, and then people want to add await later if/when there's a legitimate justification, we can. It won't break any code. But I personally doubt there will be a justification, await solves a different set of problems.

I'd be in favour of some static functions on the Promise object (e.g. Promise.then ) and ban await for now with an option to add it later if there's ever a need.

[bakkot]

I don't think this is the right thread to revisit await. As a committee member, though, I agree with the assessment that it's something this proposal needs to handle if it's to get through; if you want to discuss that further, I think a new issue would be appropriate.

[mAAdhaTTah]

Personally, I think part of what makes the pipeline operator so elegant is its simplicity. I'd be extremely happy simply writing code like this:

anArray
  |> x => pickEveryN(x, 2)
  |> x => filter(x, f)
  |> shuffle
  |> x => map(x, f)
  |> Promise.all
  |> then(xs => xs.forEach(console.log))

This builds on syntax I'm already familiar and comfortable with (arrow functions), themselves already beautifully simple and elegant, and it feels like a natural evolution for them. Introducing both the |> and a pipeline-only placeholder syntax feels like... too much. I don't think the extra syntax is needed, and I think we'd be better off treating that as a different issue.

I still think I prefer inlining await into the pipeline, despite ASI hazards, but in the interest in exhausting all options, what about maintaining the elegance of the current approach, but use the placeholder syntax to "solve" await:

anArray
  |> x => pickEveryN(x, 2)
  |> x => filter(x, f)
  |> shuffle
  |> x => map(x, f)
  |> Promise.all
  |> await ?
  |> xs => xs.forEach(console.log)

such that the placeholder is only used in await in a pipeline? For the most common case (sync pipelining), we still get a great syntax with arrow functions, and still looks pretty clean in the async case, and perhaps could be useful as a "foot in the door" for placeholders without requiring fully implementing that proposal, which would give developers an opportunity to start developing an intuition for it.

Thoughts?