MatthewKernerMS

The Confidential Consortium Framework (https://ccf.dev/) is an open-source framework for building highly available stateful services that leverage centralized compute for ease of use and performance, while providing decentralized trust. It enables multiple parties to execute auditable compute over confidential data without trusting each other or a privileged operator.

The feedback from customers of CCF had been that building apps was easy but managing the infrastructure and framework was not. This preview provides a fully-managed CCF deployment where Azure handles all infra and framework tasks so that developers can focus on their apps, without any change in the trust model.

Thanks very much for the product feedback. I've routed the points of feedback to the teams who own the components involved to take a look.

As above, I've shared these points as well with teams who own these areas. Thanks again!

Thanks much for providing specific feedback on areas where Azure could do better. This is really helpful and I've followed up with owners for the components you mentioned. None of these points fall directly into areas that I work on, but I'll make sure teams are aware.

I did want to correct one misconception, which is that we do not outsource Azure product development (we do, from time to time, hire contractors to add force on projects, but these projects are led by FTEs who are involved on a daily basis). There are certainly areas where we strive to target lean MVPs so we can get new products into customers' hands faster and get feedback earlier. I'm sure there will be some sharp edges there and we greatly appreciate you giving these services a try.

Coco doesn't use any underlying network - it forms the bottom-most layer of a permissioned blockchain network. When a ledger (could be any ledger, Ethereum as in the demo or otherwise) is integrated on top of it, the ledger decides whether to expose metadata or not, and to whom.

Maybe I misunderstand your question?

There are three layers at work in the confidentiality demo.

Coco is at the bottom of the stack. Coco handles replication of data across a network of TEE-enabled nodes, ensuring confidentiality and integrity of the data. The data is only in the clear inside each enclave. Coco enforces network-level authentication, restricting connections to actors authorized by the network constitution. At this level there are no smart contracts.

In the middle of the stack is the ledger. In the demo that is Ethereum ported to run on Coco. We disabled a few Ethereum APIs so that transactions and smart contract state cannot be read directly. We also added access control to APIs that expose events.

At the top of the stack sit smart contracts that are hosted on the ledger. Here the DApp developer can add public read methods to expose any data that they want to expose, and they can implement access control within these smart contracts to restrict access to appropriate counterparties. This can be based on RBAC with arbitrarily fine-grained rules.

In the case of bitcoin, some design would be required. Today, the scripting language is used to decide whether to unlock UTXOs at the time that they are consumed as inputs to a new transaction. If bitcoin were integrated with Coco, there would have to be some modifications made to apply access control somewhere. This could be done in the ledger code. For example, it could only allow read access to a given transaction by the counterparties (it is not obvious how the counterparties would be identified, but in the P2PKH case perhaps the caller could sign the request with the private key corresponding to the public key hash of either an input or an output). Or the bitcoin ledger could be augmented by allowing the creation of scripts to govern access. They would need to be accommodated in the transaction format and stored along with the ledger. This would be a bigger change obviously.

The technical whitepaper clarifies some of these points. To summarize, Coco networks are permissioned. Every authorized member can run a node, and all nodes see all data on the ledger. The code running inside of TEE-protected enclaves on each node determines what subset of the ledger data to expose to each caller, based on the permissions they are granted. Coco enforces basic network-level authentication, but the ledger and the smart contracts on the ledger enforce all of the app-level authN/Z.

We were talking about this yesterday. We should call them Cokens :)

Caesar consensus, which is referenced in the Coco technical whitepaper, seeks to address the problem you mention.

We tried to address several distinct concerns with Coco: scalability, latency, confidentiality and governance. Scalability and latency are determined largely by the consensus model of the network. Our intent with Coco is to make consensus pluggable so that each network can make its own choices (via the Coco network constitution) about how to run their market.

I agree with you that the branding is tricky, in part because the dominant term "blockchain" is describing a specific data structure. The key point we wanted to convey with Coco is that we are trying to enable secure, performant, multi-party computation. We think there will be many models for this over time as TEEs come into wider use. "Blockchain" is just one of them.

Many use cases we encounter in customer discussions can best be solved by using traditional databases which benefit from academic and industry investment over more than 50 years. Databases offer more mature algorithms, implementations, toolsets, talent pools and ecosystems.

That said, there are some compelling scenarios that customers are exploring where a decentralized model opens up new market opportunities, or reduces risk or reduces cost.

One challenge with a database is answering the question of who operates it, and whether that party could choose or could be compelled to act maliciously. Coco and its ilk are specifically designed to address that concern, with tradeoffs in other dimensions as mentioned above.

We tried to address several separate concerns with Coco: scalability, latency, confidentiality and governance. TEEs can be helpful in addressing each one. We are working with ledger partners who want to use Coco to address some but not all of the concerns - for example they want fine-grained confidentiality, but they don't want to rely on the TEE for consensus. In this example Coco can be used with Proof of Work.

In the technical whitepaper, one of the things we discuss is that the "Ledger Model" (data structures, APIs, abstractions and programming model) are owned by the ledger, not by Coco. I think you are right that using TEEs opens up the alternatives to some of the data structures and abstractions in use in the public networks today. We look forward to seeing what people will do with Coco in this regard.

Coco enables the ledger to mask all data, including metadata precisely to prevent the leakage you ask about. When a ledger integrates with Coco it decides what capabilities to take advantage of - and how much data to expose to the public, to permissioned participants, to counterparties or to any other role. Similarly, when someone writes a smart contract on top of a ledger, they can take advantage of the confidentiality features to decide what to expose to their callers based on the role of the caller.

We are aiming to enable a decentralized system that behaves the way that enterprises expect the data layer should behave - to only expose data to authorized callers. The cool thing about it being decentralized is that you don't have to trust/pay any one party to run it, and your authorized callers can include many counterparties within a market.

Richard Gendal Brown has a great post on the difference between a distributed database and a distributed ledger: https://gendal.me/2016/11/08/on-distributed-databases-and-di...

The Coco team shares Richard's view that the distinguishing factor is where the trust boundary exists within the system. In the case of Coco, we assume a lack of trust among consortium participants, but we leverage the attestation and anti-tampering features of Trusted Execution Environments (TEEs) to establish trust between the enclaves: assuming that the TEEs themelves are trustworthy, the TEEs can provide cryptographic proof of the software and configuration running on each enclave. In other words, I don't trust you, but my enclave has decided it can trust your enclave based on mutual attestation exchange and mutual authentication. In other words, we've transitioned from a byzantine failure mode (adversary can replace the expected remote code with arbitrary code at will) to a crash failure mode (adversary can shut the remote enclave down at will, but not alter what runs on it).

Once there is trust between enclaves, Proof of Work seemed inefficient as a consensus mechanism, although it's certainly one choice that is available and that can be used with Coco (in this case Coco would provide governance and confidentiality, but scalability and latency would be limited by PoW). Instead we can use any one of many distributed systems techniques such as Paxos or Raft to achieve consensus.

You captured the basic idea of the TEEs well. One way of thinking about this is that proof of work and proof of stake consensus protocols establish trust on a per-block basis. By leveraging TEEs, we have hoisted that trust establishment out of the per-block workflow and instead have the option to pay the cost once, at the time the network is established. One tradeoff of this design is that a TEE compromise would have major impact on the ledger. The whitepaper (https://aka.ms/cocopaper) contains a section on mitigating TEE compromise. Additionally, our intent is to support multiple consensus models, including models that require per-block trust validation. Ledgers and consortiums are free to make choices about what they want to support depending on their threat model and the assets being protected - with the ability to trade off scalability and latency against security as they see fit. What we demo'ed is one design that we think is interesting for many use cases we've heard about from customers.

Permissions are not tied to AAD members per se - we rely on a set of X.509 certs that correspond to the identities of the actors on the network for network-level authentication. These certs are referenced in the network constitution.

Ledger- and app-level authentication can be done however developers would like to do it. It can be, but need not be, AAD. See my comment above or the whitepaper for more details on the confidentiality design.

I work in Azure, and my team builds the Coco Framework. There is more information on the details in the technical whitepaper here: https://aka.ms/cocopaper

Coco is designed for permissioned networks (as opposed to public networks). The network constitution in Coco contains a list, arrived at by a vote of consortium members, of all of the actors permitted to interact with the network. Coco implements network-level authentication to restrict access to participants. The constitution state is stored within the state replicated by the Coco Framework, so it is stored on all of the nodes in the network.

Beyond network-level authentication, there is also ledger and DApp level authentication. The supply chain demo demonstrates this capability. For the demo, we modified Ethereum to integrate it with the Coco prototype. We removed the capability to read raw transactions or to access raw smart contract state directly. Instead, the DApp must implement public smart contract methods enabling reads to state that the developer wants to expose. Those read methods contain code (in this case implemented in Solidity, executed by the EVM) that is responsible for authentication and authorization of callers. This enables the DApp developer to decide for themselves now to authenticate the caller, and what roles and privileges apply to the DApp's data. We also did some work to restrict access to events to only authorized callers - you can read more in the whitepaper on this.

We believe there's an opportunity for common patterns here (a la the ERC20 token standard), but these patterns can vary by ledger and by DApp. All of that is above the Coco layer. This is why the ledger (in this example Ethereum) matters. The ledger exposes capabilities that either natively, or through smart contracts, secure the data in the ledger. Coco's job is to ensure that ledger data is only ever in the clear inside the TEE, where the ledger and smart contract code can implement a confidentiality model appropriate for the use case.