nikhilsimha
Somewhere in there is a definition of commoditization.
It is used to serve features/context to ML/AI models and rule engines, along with app views.
We are more scale and perf oriented as project given the use-cases and less decentralization oriented.
We also ship with a dataframe / prql like declarative api.
the idea is to observe with clarity the counteraction and let the sharp sensation arise and pass without the counteraction/resistance.
It could very well be that the internal mechanism of our thought has an auto-regressive reasoning component.
With the full system effectively "combining" short term memory (what just happened) and "pruned" long-term memory (what relevant things i know from the past) and pushing that into a RAW autoregressive reasoning component.
It is also possible that another specialized auto regressive reasoning component is driving the "prune" and "combine" operations. This whole system could be solely represented in the larger network.
The argument that "intelligence cannot be auto-regressive" seems to be without basis to me.
> there is strong evidence that not all thinking is linguistic or sequential.
It is possible that a system wrapping a core auto-regressive reasoner can produce non-sequential thinking - even if you don't allow for weight updates.
For example - max over {4, 5} is 5. Now I update the 5 to a 3, so the set becomes {4, 3} with a max of 4. This seems to imply that the z-sets would need to store ALL the values - again, in their internal state.
Also there needs to be some logic somewhere that says that the data structure for updating values in a max aggregation needs to be a heap. Is that all happening somewhere?
i do think ip infringement is not cool in general - but it doesnt seem right that geo research is private property.
datafusion is apache v2 licensed. [2]
pg_lakehouse built on top of data fusion is AGPL v3 + business licensed. [3]
[1] https://github.com/duckdb/duckdb/blob/main/LICENSE
[2] https://github.com/apache/datafusion/blob/main/LICENSE.txt
[3] https://github.com/paradedb/paradedb/blob/dev/LICENSE
Most companies won't touch AGPL v3 license. Maybe not a bad thing, but FYI.
I agree that my statement would be much better if used snowflake schema instead.
We have plans to implement this behavior for computing the batch arm of feature serving.
To evaluate if a feature is valuable, you could attach the value of the feature to past inferences and retrain a new model to check for improvement in performance.
But this “attach”-ing needs the feature value to be as of the time of the past inference.
The user might say - I need a refund. The bot needs to know contextual information - order details, delivery tracking details etc.
Now you have written a prompt template that needs to be rendered with contextual information. This rendered prompt is what the model will use to decide whether to issue a refund or not.
Before you deploy this prompt to prod, you want to evaluate its performance - instances where it correctly decided to issue or decline a refund.
To evaluate, you can “replay” historical refund requests. The issue is that the information in the context changes with time. You want to instead simulate the value of the context at a historical point in time - or time-travel.
Tecton, we evaluated, but decided that the time-travel strategy wasn’t scalable for our needs at the time.
A philosophical difference with tecton is that, we believe the compute primitives (aggregation and enrichment) need to be composable. We don’t have a FeatureSet or a TrainingSet for that reason - we instead have GroupBy and Join.
This enables chaining or composition to handle normalization (think 3NF) / star-schema in the warehouse.
Side benefit is that, non ml use-cases are able to leverage functionality within Chronon.
OLAP systems are fundamentally designed to scale the read path - former approach. Feature serving needs the latter.