I'm not sure if I catch your drift, but if I do:

    public <T, Z, F extends Function<? super T, Z>> Z apply(Function<F, Z> target, F fn) {
        return target.apply(fn);
    }

    @Test
    public void test() throws Exception {
        Stream<String> stream = Stream.of("foo", "bar", "baz");
        stream = apply(stream::flatMap, s -> Stream.of(s + "_a", s + "_b"));
        System.out.println(stream.collect(toList()));

        Optional<String> bean = Optional.of("bean");
        bean = apply(bean::flatMap, b -> Optional.of("soy"));
        System.out.println(bean);

        CompletableFuture<String> future1 = CompletableFuture.supplyAsync(() -> "hello");
        CompletableFuture<String> future2 = apply(future1::thenCompose, string -> CompletableFuture.supplyAsync(() -> string + " world"));
        System.out.println(future2.get());
    }

Output:

[foo_a, foo_b, bar_a, bar_b, baz_a, baz_b]

Optional[soy]

hello world

This is interesting stuff.

I'm trying to figure out how to refer to flatMap directly from the abstract function, e.g.

    flatMapTwice(x, fun) {
        return x.flatMap(fun).flatMap(fun);
    }

or

    chain(x, fun1, fun2) {
        return x.flatMap(fun1).flatMap(fun2);
    }

It should also preserve the type information (just like you did above). I don't want the caller to need to downcast from FlatMappable.

It doesn't need to be x.flatMap(f) - I'd be interested seeing a flatMap(x, f) version.

Sure:

    public interface FlatMappable<T> {
        <U> Optional<U> flatMap(Function<? super T, ? extends Optional<? extends U>> mapper);
    }

    public class FlatMappableOption<T> implements FlatMappable<T> {
        private final Option<T> value;

        public FlatMappableOption<T>(Option<T> value) {
            this.value = value;
        }

        <U> Optional<U> flatMap(Function<? super T, ? extends Optional<? extends U>> mapper) {
            return value.flatMap(mapper);
        }
    }

    // similar wrappers for CompletionStage and Stream

Now whenever you want to pass a Stream/CompletionStage/Option to something that accepts FlatMappable, you either a) already has it wrapped in FlatMappable, so pass that wrapper in; or b) has a naked value of known type, so wrap it in the propper wrapper class and pass the resulting wrapper in.

Of course, it would be nice to have this wrapping done automatically instead of manually, but that's pretty much how it's actually implemented behind the scenes (dictionary passing, or look at Go's way to wrap things inside interface{}).

Also I don't understand why would you need HKT here. I don't see language limitation here, I see poorly designed inheritance in the standard library. They should've introduced "FlatMappable" type and make Option and Stream extend it. CompletionStage, I'm torn on it, on one hand yes, it is a flatMap essentially but semantics is a bit different...

Oh my god, there's more to that, Optional is a class, not an interface and it's final :facepalm:

I've made my peace with those aspects of the libraries. I'm happy to wrap the standard classes in MyOption, MyStream, etc. Then I can extend FlatMappable myself. I just don't know how to declare and implement FlatMappable in a sane way (preserving type information, not needing casts, etc.)