ctesh

Macrometa claims it can run your microservices as functions and container running concurrently across 175 cities globally with shared durable state if you use their Global Data Network (GDN) as a globally distributed back-end platform.

They provide a global data mesh with a KVs store, a JSON Doc DB, Graphs, pub/Sub, Event Stream Processing, Complex Event Processing, Search along with edge functions and containers. Used by large enterprise customers like Verizon and twilio as well startup and fast growing SAAS players like DevCycle and Resulticks.

CEO and co-founder here. Couple of things to add to Elof's post above - (1) Macrometa is now in year 4 - has grown for a handful of paying customers to about 80 in the last 1.5 years. Counts Akamai and Cloudflare as partners along with customers across the spectrum of size - enterprise (like Verizon, Twilio to name a few larger ones) to super interesting SAAS and developer tools like Taplytics/Devcycle (https://devcycle.com/)

The distributed datamesh featuring a global KV Store, JSON DocDB, Graph DB, pub/sub, search and complex event processing is based on an novel CRDT framework and protocol - published research and papers prior to starting commercial sales at High Performance Transaction systems in 2019 - http://www.hpts.ws/papers/2019/macromedia.pdf

Hi Ignoramus - founder and CEO of Macrometa here - regret that our first attempt at explaining our consistency model caused confusion last year. Here's a link to the research paper that describes our architecture and consistency model.

https://bit.ly/HPTS-Macrometa

We got accepted in High Performance Transaction systems last year for the innovations around CRDTs for strong eventual consistency (SEC) with low read and write latencies.

Im trying to figure out how to provide a simple light weight way for fly.io users to use our global DB in their apps. It would allow a full stack to run at the edge with the compute on fly.io and the data on Macrometa either directly on fly.io or a nearby PoP (same city). Will update

Try it - https://4uiu9z43qj.execute-api.us-west-1.amazonaws.com/defau...

tl/dr - Barker is a twitter clone that runs its backend across 25 global data centers. The post explains the motivations and challenges that inspired the developer to build barker and how he leveraged Macrometa - a geo distributed platform's document database, graph database and streams to build the app. Source code provided.

Macrometa offers a free developer platform for building apps and APIs like barker at www.macrometa.co

Hi gatherhunterer Great feedback - I'll add a few links on to the docs on architecture and things like consistency of data (short answer is this a geo distributed data platform with consistency gaurantees).

Thanks also for the great point you made about "altering source of truth" - we will look into how best to provide those gaurantees as well. Our underlying data structures are an event source on an append only log - so we can go back to see the provenance of changes to make sure there was no cheating but we dont expose that externally.

A repl.it repo with a python example of a simple database app (an address book) that stores and queries data from Macrometa's serverless geo-distributed database running in California, Virgina, Germany and India.

TL/DR: Macrometa is a geo-distributed CRDT database and serverless function run time with the following capabilities 1. 25 global locations - across AWS, GCP, telecom and CDN PoPs 2. Local Write and Read performance - your query executes at the closest location to the request 3. Strong consistency guarantees (Strict ACID like for reads after writes to same loction - time bound staleness for reads to different location after write to a specific location) 4. Automatic Concurrent conflict handling - only changes to the same field of a JSON document can have conflicts and a winner is picked globally for the conflict. Changes to different fields in the JSON doc are merged. 5. Multi model - Key/Value, JSON Doc, Graph, Stream, Time series etc 6. Automatic REST API generation for queries to collections - write a query and expose it as a REST end point - easier than graphQL - can mutate data. 7. Write serverless functions to be triggered by changes to Data in streams or database 8. Free 4 region developer account - no strings attached - https://www.macrometa.co/start

Its a cloud service and is fully managed - we provide the servers, storage, networking, connectivity - everything. You just need to bring your code.

Hi PKMishra - Paid premium tiers will be introduced in July. You can request "whisper pricing" which is discounted by opening a free developer account (https://www.macrometa.co/start ) and using chat to talk to JP . He can send you the information you need.

The TLDR Macrometa is a geo-distributed, real-time, streaming DB with a multi-modal interface (K/V, Doc, Streams, Graph, Time Series). Unlike centralized eventually consistent DBs, Macrometa is a natively geo-replicated DB that uses CRDTs to replicate with better consistency guarantees than eventually consistent DBs.

The DB incorporates streams and streams processing as a first-class database primitive and lets you build low latency, high volume data pipelines that are event-driven and reactive.

Shares some feature overlap with dynamodb (key/value and document dB interfaces). Where we differentiate - global replications across all our 25 global POPs (50 by end of 2019). Integrated graphQL generator (rest as well), real-time: dB will notify clients of changes to data I.e. no need to poll, rightly integrated with streams and pub/sub, run functions and containers as triggers or stored procedures to the DB, geo query: query by lat/long/height, elastic search integrated (July 2019). There’s more - will announce in April

You bring up a great point. One absolutely can use us as a firebase alternative. We have a few customers using us with lambda and container services directly on their favorite cloud provider in just 2 locations for high availability of their apps.

The edge computing position is intentional in that we think the edge is a way to build globally distributed apps and apis. Our goal is to make as simple to build and run an app in 25 regions as it is to build an app against say firebase or dynamodb.

Yes - you hit the nail in the head - that is indeed how SPOT and regular collections work.

We think we provide the flexibility of both very strict acid behavior using spot collections Along with the strong eventual consistency model for everything else. Important to understand that as a developer you don’t need to deal with any of it - it’s handled for you once you have marked a collection as a spot.

This is not related to gun in any way. We love what gun does and it’s approach and they do some very clever things (including work on the end device). We sit as a back end database as a service in 25 global pops and process dB operations (and code expressed as functions or containers) at the closest location (by latency or geo physical location) to the user or device using an app or api rubbing against us.

we wrote our own operational CRDT engine And streams to solve this.

Thanks for pointing out the ambiguous bits - I’ll clean it up in the next few days.

Yes it does solve the double spend - it let’s you mark (check box) a collection with a SPOT property (single point of truth) which restricts the number of edge region that collection is replicated across. Additionally the spot collection is replicated across two separate data centers (and separate availability zones in those data centers) to provide high availability. The developer doesn’t need to deal with the complexity of where and how this is done - they continue to access the collection like a regular one and the DB will redirect the queries (and do things like joins) between the regular and spot collection.

I’m writing a faq and will include your questions there.

Jepsen is in the plans - we hope to get a report out in the summer.

Thanks for the truly thoughtful questions - I will put together a short faq and post a link and a a tl/dr answer here.

We also have a more technical internals paper almost completed and will publish next week.

Thanks again! This is such a helpful comment.

Hello and greetings the_arun.

The short answer is yes. Macrometa integrates a function as service (FaaS) which can be hooked into the database and be triggered by events on a stream or a data collection.

So you can for example do the following: Expose a RESTful or GraphAPI (included deep nested queries in graphQL) for one or more collections - when mutating, attach a validation function to the collection as a trigger that is called before the mutation is applied to the DB. You can also have a trigger that calls a function after the mutation is complete.

One can also do this on streams with functions being triggered to a specific topic.

Lastly - there is full support for running containers as well and you can use the endpoints exposed by the container as a trigger.

Oh and one more thing - the dB is real time. It will notify clients of updates to collections automatically (like firebase).

Hope this helps..

ok Take 2:

1. We use causal consistency using vector clocks to establish the causal order of operations instead of using Time stamping and Network Time Protocol which are unreliable over WAN distances

2. We share Database state changes/updates between each Point of Presence (POP) to the others in the cluster using Asynchronous Streams that use a pull model instead of a push to maintain ordering of messages over the network. This has the added benefit of allowing us to know exactly how current each PoP is with changes.

3. We dont use Quorum for inter PoP to establish consistency - there's a white paper on our site that shows you the how and why of coordination free replication. Gist is we have developed a generalized operational CRDT model that allows us to get Associtive, Commutative, Idempotent convergence of changes between PoPs in the cluster without needing quorum

4. the DB is a multi master - you can access and change data at any PoP. Its also multi model and lets you query your data across multiple interfaces such as key/value, documentDB (JSON), Graph etc

5. the DB automatically creates GraphQL and REST APIs for your schema taking away the complexity and effort of a lot of boilerplate development on the backend

6. The DB is available as a managed service in 25 PoPs today - you can request an account and we will give you one. WE will be generally available with a free tier in April and you can signup online and self administer your cluster

7. You can access the DB via a CLI, GUI, or write code that accesses it via REST, GrapHQL or using native language drivers in JavaScript, Python today (we are working on other languages with a view on releasing them over the next few months)

hope this helps...

The TL/DR on how:

1. Causal consistency with the ability to create collections (tables) with strict consistency - uses vector clocks and not wall clock timestamps for ordering DB operations

2. Streams to propagate DB changes from one geo location (node) to another with guaranteed ordering and reliable delivery

3. Generalized Operational CRDTs to make all DB operations Associative, Commutative, Idempotent and Distributed (the new ACID)

4. One data model - multiple interfaces - store and query your data as Key/Value DB, DocumentDB, Graph DB, StreamDB, Geo location DB (query by lat/long/altitude)

5. Automatic GraphQL and REST api generation for your schema

6. Multi master - query and update any of our 25 global locations and get one consistent view of data globally

7. Access via CLI, REST, GraphQL (built in server), client drivers for python, Javascript today (Java, Go, Ruby in the works)