API or documentation?

[williamcotton]

Is there an overview of the architecture for ChromaWay colored coins?

What's the relationship between all of these different projects: cc-wallet-engine, coloredcoinjs-lib, chromawallet-proto, cc-wallet-core

Are there demos of these projects running on a website somewhere?

[killerstorm]

We were planning to add documentation a bit later. But, yes, there is an overview, it was posted on reddit:

And thus we implemented modular JS libraries which form a complete colored coins wallet, but can also be used in a variety of applications involving colored coins.

There are three libraries offering different levels of abstraction:

1. coloredcoinjs-lib is low-level. You can ask it about colorvalues of transaction outputs, and it can also construct transactions, but you have to provide inputs yourself. This is probably what you should use if you want add colored coins support to an existing wallet.
2. cc-wallet-core implements a complete wallet, taking care of everything from keypairs to pushing transactions to the network.
3. cc-wallet-engine offers a high-level interface on top of cc-wallet-core. It takes care of low-level details, such as amount formatting.

Each library uses the more low-level one under the hood, e.g. cc-wallet-engine uses cc-wallet-core. And chromawallet-proto is a sample UI built on top of cc-wallet-engine.

There is a demo, but it's a bit outdated: http://killerstorm.xen.prgmr.com/alex/cw/demo-eng.html (and a faucet is very slow because it uses outdated libraries.)

You can easily build this demo out of chromawallet-proto.

Please let me know if you need more info.

[williamcotton]

A pattern that I've found useful with Bitcoin metatransactions protocols (like OpenAssets, BlockSign, etc) is to have a client that can build complete unsigned transactions when provided with these basic building blocks:

someProtocol.build({
  options: { ... },
  address: address,
  unspentOutputs: unspentOutputs
}, function(error, builtTx) {
  console.log("an unsigned built transaction", builtTx);
});

or even going further when provided with a function that will sign and propagate the transaction as well:

someProtocol.buildSignAndPropagate({
  options: { ... },
  address: address,
  unspentOutputs: unspentOutputs,
  propagateTransaction: propagateTransaction,
  signTransaction: signTransaction
}, function(error, receipt) {
  console.log("the receipt after building, signing and propagating the tx", receipt);
});

Another interesting approach is taken by the CoinPrism API. It is a REST interface that builds and returns unsigned transactions to wallet software.

You can see examples here: http://docs.coinprism.apiary.io/#reference/transaction-builder/post

Does or will coloredcoinjs-lib support this kind of interaction? Right now it seems like some of this functionality is contained in cc-wallet-core.

The main point is that the best interface I see for these kinds of Bitcoin metatransaction protocols is one where the client libraries do not control or manage private keys. That is, that they do not function as wallets, merely as transaction builders with perhaps support for helper functions that will call out for signing and propagating.

Also, can you point me to some ChromaWay colored coin transactions as they exist on tesnet3 or the Bitcoin mainnet? I'd love to see some live examples. Even better would be a description of that transaction that breaks down how it works!

BTW, I'm building a blockchain metadata browser. Here's a screenshot:

I'd love to be showing ChromaWay transactions as well!

BTW, the tool also authors these transactions. So far just OpenAssets and BlockSign but I'm adding them as quick as I can. I'd love to be able to build ChromaWay transactions with my own wallet

Right now I'm still discovering the best interfaces for each of these metatransaction protocol types. I'd like for there to be a standard protocol. I live in San Francisco and have started reaching out to everyone in hopes of getting this discussion started.

Does anyone on your team live in the Bay Area?

[williamcotton]

Also, do you have a ChromaWay blockchain browser like CoinPrism?

https://www.coinprism.info/tx/4309b7f2bb85f18dac820d4e6d38b73110b4b28c61364a14b979d31890b03a99

[williamcotton]

And where exactly do you store your metadata? Not in OP_RETURN?

[killerstorm]

Another interesting approach is taken by the CoinPrism API. It is a REST interface that builds and returns unsigned transactions to wallet software.

This requires you to trust the server, as it can give you a bad transaction to sign.

Does or will coloredcoinjs-lib support this kind of interaction?

Yes. but it's quite a bit verbose.

1. You need to implement OperationalTx which acts as an interface between library and the wallet
2. Then feed OperationalTx instance to EPOBCColorDefinition.makeComposedTx
3. Which gives you an instance of ComposedTx, which you can easily convert to the transaction proper.

The main point is that the best interface I see for these kinds of Bitcoin metatransaction protocols is one where the client libraries do not control or manage private keys.

True, that's why we implemented coloredcoinlib which is fully separate from the wallet.

Also, can you point me to some ChromaWay colored coin transactions as they exist on tesnet3 or the Bitcoin mainnet?

OK, I'll dig one up.

Does anyone on your team live in the Bay Area?

No.

Also, do you have a ChromaWay blockchain browser like CoinPrism?

We had at one point, but I think it no longer works... are planning to implement another one at some point.

And where exactly do you store your metadata? Not in OP_RETURN?

There are only 12 bits of metadata per transaction, it is stored in nSequence of the first input. Otherwise, colorvalues are encoded using satoshi values.

[williamcotton]

This requires you to trust the server, as it can give you a bad transaction to sign.

The transaction can still be parsed and inspected by the client before signing so I don't really see that as an issue.

Client software that creates transactions need to trust the client libraries that they use. Of course this could be done with just a single audit as opposed to having to check every transaction as with a REST interface.

However, don't make assumptions on how much trust developers require from your protocol or implementations. There will always be varying needs.

The important thing about a REST interface for a building transactions is that it makes it easier for developers to work with. Protocols that are not developer friendly won't be used as much as protocols that are.

Right now it is easier to view, understand, scan and create OpenAsset transactions than any other colored coin protocol.

There are big limitations with OpenAssets and I would definitely like to use colored coin protocols that are capable of non-reissuable coins and tokens but because OpenAssets has the best blockchain explorer, developer tools and documentation I was more inclined to implement that protocol first.

Again, I'm interested in supporting any and all forms of Bitcoin metatransactions.

Yes. but it's quite a bit verbose.

Make an easy-to-use interface for this and you'll incentivize developers to use your tools!

[killerstorm]

FWIW coloredcoinjs-lib is supposed to be compatible with every imaginable colored coin protocol. E.g. if you implement OpenAssetsColorDefinition it will able to understand OpenAssets colors.

Make an easy-to-use interface for this and you'll incentivize developers to use your tools!

cc-wallet-engine is easy to use. Well, I see your point, but I think somebody who is implementing a wallet should be able to afford a couple of hours to understand how API works.

[williamcotton]

FWIW coloredcoinjs-lib is supposed to be compatible with every imaginable colored coin protocol. E.g. if you implement OpenAssetsColorDefinition it will able to understand OpenAssets colors.

How many colored coin protocols are currently supported by coloredcoinjs-lib?

Well, I see your point, but I think somebody who is implementing a wallet should be able to afford a couple of hours to understand how API works.

The solution that you described involving OperationalTx, EPOBCColorDefinition.makeComposedTx and ComposedTX are internal systems that I know nothing about. I would have to blindly study your source code to figure this out. This will definitely take more than a couple of hours! Good and useful public APIs don't need to expose this internal functionality.

Perhaps you could make some documentation or examples that show how this would work? Perhaps when I have some time I could fork one of these projects and implement the kind of interface that makes sense for my needs.

I think you'll find all of these kinds of APIs would benefit from the assumption that colored transactions will be signed by a system that manages it's own private keys. This makes the API easier to consume by existing wallet software.

[killerstorm]

How many colored coin protocols are currently supported by coloredcoinjs-lib?

Just one. We implemented ~5 different color kernels in the Python version of coloredcoinlib until settled on EPOBC.

are internal systems that I know nothing about.

These aren't internal systems, it is an API. E.g. OperationalTx is an interface which describes which functions coloredcoinjs expects wallet to provide.

I.e. instead of providing 10 parameters and options, you just provide an instance of OperationalTx which encapsulates all of this stuff.

It's just called in a weird way, I guess, for historic reasons. Should be something like WalletInterface or Context, perhaps.

It was called OperationalTxSpec in Python coloredcoinlib, the idea was that you provide colordcoinlib a specification of the transaction you want a compose, and it composes it. coloredcoinlib was also supposed to be fully abstract of scripting and signing. It just composes inputs and outputs according to requirements of color kernel, and then the wallet constructs actual scriptSig/scriptPubKey as needed.

I agree that it's a bit too complex in case one doesn't need all this flexibility, so if there is a demand for it, it would make sense to make a simplified API on top of raw functions.

Perhaps you could make some documentation or examples that show how this would work?

Yes, that's the plan.

I think you'll find all of these kinds of APIs would benefit from the assumption that colored transactions will be signed by a system that manages it's own private keys. This makes the API easier to consume by existing wallet software.

This is why we're providing coloredcoinlib. It doesn't even have a notion of signing a transaction.

And it's fully abstract of scripting too. Whether you wallet uses plain P2PKH, P2SH, multi-sig or steal addresses, we can work with it. Even if you need complex scripts to implement trading across chain contract, you can still use stock version of coloredcoinlib to compose transactions involving colored coin cross-chain trade.

It doesn't seem to be possible with OpenAssets API, as it seems to be able to have a fixed repertoire of transactions it can do. Their strategy was to start with something simple (send colored coin to a single address) and add more and more complex transaction types.

But we implemented an API which can compose any kind of a transaction and factored all irrelevant parts out of it.

[williamcotton]

It just composes inputs and outputs according to requirements of color kernel, and then the wallet constructs actual scriptSig/scriptPubKey as needed.

What format are those inputs and outputs in? Is it portable?

I agree that it's a bit too complex in case one doesn't need all this flexibility, so if there is a demand for it, it would make sense to make a simplified API on top of raw functions.

Perhaps there is a demand for the services provided by your APIs but the price of entry is too high! A simpler API for developers is a less costly API for developers. The cost is in the time it takes to familiarize oneself with the new API.

A great side effect is that the documentation for modules with simple interfaces practically writes itself! The associated tests will tend to be very descriptive examples of how to use the module.

Have you see modules that ship with tests with integration tests that interface with testnet3?

Helloblock.io has a test faucet and testnet API that bitcoinjs-lib uses for an incredibly well presented example of how a multi-signature transaction is built and used.

https://www.npmjs.com/package/helloblock-js
https://github.com/bitcoinjs/bitcoinjs-lib/blob/master/test/integration/multisig.js

'getRawTransaction' and the other Raw Transactions JSON-RPC API methods seem to be a very portable solution and natural interface.

https://en.bitcoin.it/wiki/Raw_Transactions#JSON-RPC_API

[williamcotton]

And here's an example of another OpenAssets implementation that exposes a different set of problems:

https://coins.assembly.com/docs

The server does all of the signing. Private keys are a problematic interface for client libraries.

The raw bytes of a transaction, either signed or unsigned, are the best interface point for Bitcoin client libraries. Any sort of multi-signature will also benefit from raw transactions bytecode as we can almost always assume that different signatures will come from different types of wallet implementations.

If the state of the system requires the full Bitcoin blockchain to be back-scanned we need to be building libraries that interface with bitcoind and can build and verify the state from the ground up much like https://www.npmjs.com/package/openassets. Again a natural interface seems to be the Raw Transaction JSON-RPC API. It seems like we should all build these... and if we were extra nice about it, have common interfaces to various data stores like postgres, redis, mongo, cassandra... What about a middlewear system between bitcoind and a data store?

Either this or we can choose to trust the state of public API endpoints like with https://api.coinprism.com - and there are definitely use cases that justify both approaches!

What I find interesting is that there are two product implementations built on top of OpenAssets that have incompatible metadata systems. They can see each other's transactions and trace the spends but they have no idea what the names of any of these things are. Perhaps this is fine and it is up to them to release their own client libraries. I've been treating them as two different types and hitting both of their public APIs to retrieve this metadata and get names or images of the assets.

How does ChromaWay plan to deal with metadata and will you have a public API and client libraries for this metadata?

[williamcotton]

What about a middlewear system between bitcoind and a data store? Each transaction could have an associated metatype identifier and a JSON metadata document that is built by different processors... OpenAssets, BlockSign CoinPrism, Assembly, ChromaWay, ColoredCoins, etc. So you back-scan and build the state and all metadata for each protocol storing it in a bunch of supported data stores like postgres, redis, etc.

The processors themselves should be be separate from this bitcoind interface and should still work if only for read-only remote APIs. This would allow for fully functional web browser clients that could have a websocket listening for and parsing incoming live transactions complete with associated metadata.

[killerstorm]

What format are those inputs and outputs in? Is it portable?

We treat inputs as opaque objects. Whatever format wallet uses, it gets them back.

Outputs are just (value, script) pairs, where script is hex-encoded.

The cost is in the time it takes to familiarize oneself with the new API.

Well, I would be surprised it's more than one day. At least when one sees usage examples.

Have you see modules that ship with tests with integration tests that interface with testnet3?

We have integration tests of this kind for cc-wallet-core.

How does ChromaWay plan to deal with metadata and will you have a public API and client libraries for this metadata?

Well, I believe that doing it in a layered fashion is the best way to do it. Low-level coloring level is fully agnostic of metadata: it doesn't care what kind of an asset is being transferred, it works with color identifiers.

Meta-data appears on higher level. Let's say if somebody is building a stock market, he needs metadata specific to stocks. (And within our libraries we have an AssetDefinition class, which concrete applications can extend in any way.)

As for discoverable meta-data, I believe it is utterly stupid and outright malicious concept. Any metadata which is published in the blockchain or other medium can be trivially duplicated.

So suppose I buy oracle-stock.com, get an SSL cert for it, register ORCL OpenAssets asset. Then I can try to sell it to somebody, and it will show up in CoinPrism as Oracle stock. Unsuspecting user might believe it's true.

We use a different model in ChromaWallet: user explicitly needs to choose to install asset definitions he trusts.

In future we're going to introduce a notion of asset directories. E.g. a user who wishes to trade stocks can go to a broker and connect to him (thus fetching a list of asset definitions), getting an ability to trade ORCL, MSFT, etc. Obviously, he needs to trust this broker. Or it might be a bank or an exchange users connects to. Important thing here is that user should explicitly trust the source of asset definitions.

And the whole list of all issued assets is definitely useless, as quite likely most of them will be junk, fake, scam or test assets. I don't think anybody is going to enjoy looking through thousands of such assets. (We actually had a global list of assets in WebcoinX... Ended up with 100+ assets people made for testing purposes.)