Infrastructure

Understand Charli3's Oracles: Learn about the architecture and mechanism for generating reliable data.

This article assumes familiarity with the (E)UTXO model. It reviews Charli3's oracle contracts and its integration with the Vasil Hard Fork features, including inline datums, reference UTXOs, and reference scripts.

First, we will examine one of the most basic oracle contracts, including its UTXOs, datums, tokens, and NFTs, as well as the mechanism that incentivizes individuals towards active participation in data acquisition. Then, we will look at a more advanced oracle contract that uses multiple trusted data sources and examine how it's UTXOs work together to provide accurate, decentralized, and trustworthy data.

Oracle Contract's UTXOs

In this section, the most basic oracle contract we will employ comprises five essential UTXOs:

1. Node UTXO (s): Facilitates the transfer of external blockchain data, such as information retrieved from various APIs, to a predefined aggregation smart contract, often referred to as an oracle contract.

2. AggState UTXO: Encompasses the configuration details of the oracle contract and the associated reward tokens (C3). This configuration empowers users to tailor a C3 network according to their specific requirements. For instance, it enables the customization of feed generation frequency, determination of reward quantities, and other features as elaborated in the Oracle Settings section below.

3. Reward UTXO: Crafted to record the received C3 Tokens as compensation for each network active member, these tokens are distributed in every aggregation transaction, transferring C3 tokens from the AggState UTXO to this specific record UTXO.

4. Feed UTXO: A UTXO that houses the consolidated and refined data derived from all node operators, known as the C3 Network Feed or Oracle Feed. It acts as the primary data source for third-party contract consumers.

5. Reference Script UTXO: Houses the contract's script, facilitating its reuse and reducing transaction costs.

Figure 1. The simplest Oracle contract creates five UTXOs at its initiation stage.

In Figure 1, the example illustrate a single Node UTXO for the sake of simplicity. Nevertheless, in production environments, C3 Networks can accommodate up to a maximum of six Node UTXOs.

Node UTXO

Let's concentrate on the Node UTXO for now. Upon executing the start oracle transaction, the inline datum will only contain information about the wallet that is authorized to consume it (Public key Hash) and a NodeFeed token. The individual, or entity, that holds the wallet access, can modify the information stored in the datum.

Figure 2. The Update-Node transaction modify the UTXO's inline datum.

As depicted in Figure 2, the Update-Node transaction empowers node operators to update information within the Node UTXO (from A to B).

Regarding the exchange rates among assets, the off-chain component of the C3 Network consolidates data from diverse APIs to derive a definitive value. An established approach for this aggregation entails calculating the median from N distinct APIs; subsequently, this N value represents the ultimate output. In Figure 2, it is plausible to interpret this N value as B.

AggState UTXO

The datum of the AggState UTXO encapsulates the contract settings, commonly referred to as oracle settings. These settings delineate the processing of information derived from the Node UTXOs and encompass parameters such as data update frequency, authorized nodes, payment rewards, platform signatories, etc.

The UTXO values are comprised of the Aggregate's NFT (AggState) along with a quantity of C3 tokens. These tokens serve as incentives for Node operators who consistently update their information; and for the platform operator, tasked with monitoring the health and correctness of the contract performance.

Figure 3. The AggState UTXO's datum and values.

Feed UTXO

The Feed UTXO is equipped with a distinctive NFT (OracleFeed) tailored to identify the oracle feed within any C3 Network. The data generated by the Aggregate transaction is meticulously stored within this UTXO, adhering to the Datum Standard. The information within the Feed UTXO seamlessly integrates with any smart contract through the read-only UTXO mechanism, effectively preventing race conditions.

Figure 4: The Feed UTXO's datum and values. The UTxO is populated with information after each aggregation transaction.

Reward UTXO

The Reward UTXO maintains distinct records for each node's earned reward, such as the compensation for the oracle operator contributing fresh and accurate information to the C3 Network. The associated NFT for this UTXO is referred to as Reward.

Figure 5: The reward UTXO's datum and values.

Network Rewards

1. Rewards for consensus contributing node operators (fresh data and accurate information).

2. Rewards that encompass the aggregate transaction costs. Each node has the capability to execute the aggregation transaction within the parameters defined by the C3 Network. The cost associated with performing this operation is included as a reward.

3. Rewards for platform operation and maintenance. It serves to cover the network’s operational expenses, which include maintenance, monitoring tools, and alert systems.

Reference Script UTXO

To minimize user fees and prevent blockchain bloat, scripts can be stored separately in the blockchain. Rather than including the complete script in each contract transaction, a spending transaction can reference the script's location and utilize it for validation when spending a UTXO. This approach eliminates the need to duplicate the script in every transaction, leading to more efficient and streamlined transactions.

Consensus algorithm with a single Node UTXO

In our simple oracle example, when executing the Aggregate transaction the contract read a single Node UTXO, and its information is reflected in the Feed UTXO through the consensus algorithm.

The Node UTXO that supplies the information is rewarded twice with a fixed amount of C3 tokens. The first reward acknowledges the accurate data provided to the oracle contract, while the second reward covers the costs associated with the Aggregate transaction.

The Feed UTXO's datum now includes the processed node's information, also known as the oracle feed (or C3 Network feed). Please refer to Figure 6 for a visual representation.

Although this example contract only involves mirroring information from one UTXO to another, it effectively demonstrates the collaborative functionality of each UTXO.

Note: To ensure decentralization and security, Charli3 does not provide contracts with a single Node UTxO. The example described introduces readers to the Aggregate transaction for educational purposes only.

Figure 6. The Aggregate transaction takes in all oracle UTXOs and calculates a new Feed UTXO's datum (A) through the consensus algorithm

Consensus algorithm with multiples Node UTXOs

For C3 Networks employing multiple Node UTXOs, such as those involving exchange rates between assets, the consensus algorithm determines the final feed by computing the median value of all nodes' exchange rates. Nodes with input datums not flagged as outliers during the consensus phase are rewarded.

The on-chain consensus phase incorporates two outlier-detection methods: the Interquartile Range Algorithm (IQR) and the Percentage Divergence Algorithm (PD). The IQR identifies outliers as values beyond the 25th and 75th percentiles. Concurrently, the PD measures each response's deviation from the median against a set threshold, utilizing data from all nodes. Imposing penalties on outliers enhances the network's reliability.

The final feed from the consensus algorithm (C3 Network feed), is stored in the datum of the Feed UTXO. Third-party contracts can access the oracle feed by identifying the corresponding NFT at the contract address.

Note: To ensure precision and accuracy for C3 Networks involving data types unsuitable for the consensus algorithm, specialized functions will be employed.

Oracle Settings

The oracle settings store adjustable variables that cater to the C3 Network focus and target audience requirements, including:

1. List of authorized wallet's Public Key Hashes (PHKs) for updating each Node's UTXOs.

2. Percentage of C3 nodes needed to generate a new C3 Network Feed.

3. Timeframe for determining the freshness of Node's datum information.

4. Time frequency for C3 Network Feed updates.

5. Percentage change is calculated by comparing the previous and incoming C3 Network Feed. If the percentage surpasses a predefined threshold, a new C3 Network feed is generated.

6. The minimum quantity required to replenish the C3 token pool, serving as a reward for entities that generate new feeds. Anyone can replenish the pool; however, a lower limit is established to prevent dusk attacks.

7. A specific timeframe is defined within which an Aggregate transaction must be submitted to the blockchain. If the transaction surpasses this designated window, it will be considered invalid. This precautionary measure is in place to prevent the acceptance of outdated data, which could be a result of system delays or network congestion.

8. C3 reward quantity for each participant in the contract, including:

   1. Amount for Node operators participating in the consensus algorithm.

   2. Amount to cover the Aggregate transaction costs for Node operators.

   3. Amount for the platform operator for maintenance purposes.

9. IQR: Value for data dispersion measurement (consensus).

10. PD: Upper data variation limit (consensus).

11. Individuals/entities (PKHs) allowed to modify the C3 Network settings.

12. The least number of authorizations signatures (11) required for modifying the C3 Network.

Settings 4 and 5 dictate the execution of the Aggregate transaction. Setting 4 triggers the transaction at a fixed frequency, while Setting 5 activates it when there is a percentage change in the oracle feed that exceeds the previous oracle feed.

The percentage deviation (5) from the previous generation C3 network feed is assessed off-chain, with a higher frequency, such as every 2 minutes as a minimum.

Exception handling

Node Downtime

In cases where certain UTXO Nodes fail to update regularly, oracle setting 3 evaluates and excludes inactive Nodes during data processing. Setting 2 allows the oracle operator to specify the minimum percentage of Nodes with recent information required to trigger an Aggregate transaction. For example, if there are 6 Node UTXOs in the oracle contract, the operator can set the requirement to have at least 50% of them providing active information.

Charli3's oracles and nodes are strategically distributed globally to ensure uninterrupted data access. The Charli3 team have implemented robust monitoring systems for each node operator backend and C3 Network promptly sending notifications in the event of any potential issue or disruption.

Outlier Detection

Cheating attempts within the system, such as providing false node information, are deterred by the consensus algorithm. Submitting an Update Node transaction to the Cardano blockchain incurs a fee, discouraging false submissions as it would lead to exclusion from the consensus algorithm.

Oracle settings 9 and 10 allow for setting a threshold for outliers and selecting values closest to the median from all participants. Consequently, only a limited number of nodes with up-to-date data, such as 5 out of 7 nodes, may receive C3 token rewards. This ensures that rewards are distributed to the most reliable and accurate contributors.

The platform operators have the capability to remove underperforming Node UTXOs by executing the Delete Nodes transaction maintaining the quality of the feed output. Similarly, the Add Nodes transaction allows for an increase in the number of participant nodes, enhancing the overall participation in the oracle network.

On-Chain Code

The Charli3 on-chain contract code ensures security by securely storing token reward information (C3) and NFTs within the validator argument (oracle data type). This information is immutable, ensuring that participants in the network cannot be subjected to cheating or manipulation.

A valid argument for a contract validator could be:

Validator's argument

oracle :: Oracle
oracle =  Oracle
      { oracleNFT =
          AssetClass
            ( "e36cd9189099d7bedae87e8702e86e38122a62141743e1a64a5349ff"
            , "OracleFeed")
      , aggStateNFT =
          AssetClass
            ( "e36cd9189099d7bedae87e8702e86e38122a62141743e1a64a5349ff"
            , "AggState")
      , rewardNFT =
          AssetClass
            ( "e36cd9189099d7bedae87e8702e86e38122a62141743e1a64a5349ff"
            , "Reward")
      , feeToken =
          AssetClass
            ( "436941ead56c61dbf9b92b5f566f7d5b9cac08f8c957f28f0bd60d4a"
            , "C3")
      , nodeToken =
          AssetClass
            ( "e36cd9189099d7bedae87e8702e86e38122a62141743e1a64a5349ff"
            , "NodeFeed")
      }

The entire Charli3 mechanism is executed in the on-chain code, making the entire process transparent. Rewards, configurations, and generation times are visible to the general public. Unlike other systems where a significant burden is placed on off-chain processes, a more transparent approach inside the Cardano blockchain has been chosen.

Reward System

The reward mechanism is designed to remunerate all network participants for their active involvement in the C3 Network feed generation. It comprises two distinct components, either of which can be used independently: the Static Payment System (SP) and the Dynamic Payment System (DP).

The SP allocates a predetermined quantity of C3 to every participant in the network. Consider a scenario where a C3 Network is established to track the ADA/C3 market price. In this example setup, each node is assigned 2 C3 tokens, with 0.8 C3 designated for the Aggregate transaction and 1.5 C3 tokens allocated to the platform operator, regardless of the prevailing market rates.

As illustrated in Figure 7, the aggregate transaction disburses this set reward amount.

Figure 7. Rewards are set in the AggState UTXO, transferred to the Reward UTXO after each aggregate transaction, and can be claimed by participants anytime.

The DPS leverages real-time exchange rates to distribute rewards to all network members. This adaptable system allows for conversions between any assets as rewards.

For instance, a network utilizing the ADA/C3 exchange rate can effortlessly translate ADA into its equivalent C3 Token value, guaranteeing that rewards mirror current market valuations. For a graphic illustration, please see Figure 8. One C3 Network might track the ADA/C3 rate conversion, while another network could leverage this data, enabling a seamless exchange between the C3 Token payment and the value of a different asset. This process not only determines the payout in C3 Tokens but also ensures the preservation of its market value.

Figure 8. The Dynamic Payment System taps into an external C3 Network to ascertain the precise market value when rewarding network participants.

Off-chain Code

Figure 9. illustrates the distribution of requests across various online sources and C3 Node sources. The generation of Feeds involves a two-level aggregation process.

Charli3’s off-chain core is essential for the C3 Network’s deployment on the Cardano blockchain. It adeptly manages tasks such as token minting, data security, transaction coordination, and transaction monitoring.

Thanks to its reliance on Pycardano, this core seamlessly integrates with other components within the C3 Ecosystem. Recognizing the inherent costs and limitations associated with on-chain operations, Charli3's off-chain core is meticulously designed to optimize cost efficiency.

To submit transactions, the off-chain component leverages two external services, namely Blockfrost and Ogmios. These services can be employed independently or in tandem, offering flexibility and adaptability. Subsequently, a secondary on-chain aggregation step is implemented to uphold the principles of security and transparency, culminating in a robust and reliable process. For a visual representation of this entire process, please refer to Figure 9.

Platform Operators transactions

The platform operators have the ability to perform several transactions, which are implemented as off-chain code in both the Haskell and Python versions of the contract:

1. Remove-Nodes: This operation allows the platform operator to eliminate specific Node UTXOs by burning the Node tokens associated with the specified wallet IDs.

2. Add-Node: The Add-Node transaction enables the creation of new Node UTXOs. The Oracle owner provides a list of wallet IDs, and additional Node tokens are minted and distributed to these new UTXOs.

3. Edit-Oracle-Settings: The Edit-Oracle-Settings transaction provides the ability to modify the datums of the Aggregate UTXO by changing the values of the oracle settings.

4. Add-Funds: Refills the C3 Token pool, thereby guaranteeing rewards for all parties within the C3 network and facilitating the generation of new C3 network feeds.

5. Platform-Collect: The Platform-Collect transaction allows the platform operator to collect the rewards earned.

6. Oracle-close: In the Oracle-close transaction, all contract UTXOs, including associated NFTs, ADAs, and C3 tokens, are collected and consumed. If the Node UTXOs contain C3 tokens, these tokens are paid to the corresponding node wallet operators. This transaction marks the end of the contract since no usable UTXOs are left.

Multi-Signature Validation

Changes in the C3 network use a multi-signature validation, needing approval from multiple parties. While some transactions bypass this, like the Add Funds transaction, others mandate agreement, ensuring network control.

Node operator transactions

1. Aggregate: Execute the Aggregate transaction to generate a new oracle feed.

2. Update-Node: The node operator can modify data in the Node UTXO, but not the wallet ID or NFT information.

3. Node-collect: The Node UTXO owner can withdraw the rewarded C3 tokens from the Reward UTXO to their personal wallet.