M_Rys_Msft
Besides Pig's extensibility story is not that good.
But I am glad you like Pig, because Pig is supported in the system as well :).
As to your join syntax: Conceptually, U-SQL is a functional lambda expression composition language. One of the major lambdas are rowset expressions. Thus in your example above a is by itself a rowset lambda, "a JOIN b ON comp" is a rowset lambda etc. What does this mean? It means that eventually, the language may allow you to just write:
a join b on a.id == b.id
instead of
SELECT * FROM a join b on a.id == b.id;
At that point the SELECT ... FROM clause becomes a projection and applier of scalar transforms and aggregators.
Today, U-SQL is not quite as flexible, but we do support this pattern already for Table-Valued Functions where you can say
TVF(42)
in addition to
SELECT * FROM TVF(42)
Good discussion though... keep it coming!
CREATE VIEW V AS EXTRACT <insert schema here> FROM "/a/b/{*}.csv" USING Extractors.Csv();
If the data is more complex, you can write your own Extractors that for example handle the divergence in columns. Although we are planning on some features here in the future.