Inside Feeders: The Decentralized Data Collectors Powering Lumina

• Permissionless Network: Anyone can run a Feeder
• Decentralized Data Sourcing: Independent data operators collect data from multiple sources
• Multi-Step Data Processing: Scrapers, Collectors, and Processors ensure accurate, aggregated data
• Economic Incentives: operators earn rewards for honest data reporting

Feeders make up a decentralized network of independent data operators that power the data collection and processing within the DIA Lumina oracle stack. These operators are responsible for fetching data directly from sources and submitting it on-chain to the Lasernet rollup.

The Role of Feeders

The key role of Feeders is to provide a decentralized approach to data collection, enhancing the security and reliability of the Lumina architecture. Each Feeder operates independently, retrieving data from selected sources and pushing it onto Pods – dedicated smart contracts associated with each Feeder in the Lasernet network.

Open Participation

Anyone can spin up a Feeder and start pushing data on-chain, permissionlessly. To participate as a Feeder, actors must stake DIA tokens in the Lasernet network. This staking requirement serves as an economic incentive to ensure honest data reporting and deter malicious actors. On the other hand, Feeders are rewarded with DIA tokens for performing the data-sourcing work.

Feeders follow a structured process to collect and submit data in 4 steps, executed by different sub-components:

Scraper

Its function is to continuously fetch granular data from sources, for example, gathering each trade tick for the cryptocurrency ETH in several CEXs or DEXs. In this case, the trade data is sent to the channel tradesChannel, where the expected input for a scraper is either a pair ticker or a set of symbol ticker as ETH-USDT.

Feeders run a dedicated scraper for each source, for example, an instance of Uniswap on Arbitrum. New scrapers are continuously developed by the DIA community and merged into the DIA GitHub repository, where anyone can access scrapers for 100+ sources freely.

Collector

The collector gathers trades from all running scrapers. As soon as it receives a signal through a trigger channel it bundles trades in atomic tradesblocks. An atomic tradesblock is a set of trades restricted to one market on one exchange, for instance, ETH-USDT trades on the Binance exchange. These tradesblocks are then sent to the Processor.

Processor

The processor is a 2-step aggregation procedure:

1. Step: Aggregate trades from an atomic tradesblock. The type of aggregation can be chosen as a free parameter and the set of available methods will be continuously extended through an environment variable (see Feeder/main). The only assumption on the aggregation implementation is that it returns a float.
2. Step: Aggregate filter values obtained in step 1, for example, build the average of ETH-USDT prices on Binance, Coinbase, and Kucoin exchange. As in step 1 the method of aggregation can be freely chosen in general. The obtained scalar value is sent to the Pod associated with the Feeder.

Pods

The Pod, a simple key-value oracle smart contract, publishes the value obtained from the Processor. It is worth mentioning that the Pods can contain the trigger mechanism that initiates an iteration of the data flow diagram.

The current Feeder implementation demonstrates just the beginning of what’s possible within the Lumina architecture. Like Ethereum’s diverse client ecosystem (Geth, Besu, Nethermind), Lumina embraces permissionless innovation – developers can create and deploy customized Feeders with unique aggregation logic and methodologies.

This diverse implementation approach strengthens the network’s resilience through methodological diversity, while enabling cross-validation across different Feeder types to ensure data integrity. The competitive nature of the ecosystem drives continuous optimization for speed and efficiency, encouraging innovation and specialization in data collection methodologies.