tde and Noble establish real-time data ingestion protocols for global fleet via Amazon S3
tde proNova, in partnership with Noble, has designed a data transmission protocol to connect rig data ingested by a client-operated central repository to Amazon S3 – the object storage service from AWS – for the purpose of delivering and advancing automation in data processing, quality control, and analytics.
Challenge
Noble stores the raw sensor data streams from the rigs in the OSIsoft PI data platform for many different channels. They requested an automated process to continuously stream the raw data for a large set of channels to proNova for drilling performance analysis, without the need to setup a standard WITSML interface in-front of their own OSIsoft PI data platform.
Recommendation
The implementation of a new data fetcher type in the sensor data processing pipeline of proNova was recommended. The OSIsoft PI Integrator application supports the automated push of raw sensor data in CSV format to a target S3 bucket. Therefore, tde suggested to implement a Lambda function that converts the received data from CSV format into a file format that can be interpreted by the data fetcher in the proNova software.
Solution
Raw drilling and completions data is now captured at the rig site from surface equipment sensors and transmitted to the client’s centralized data repository. proNova’s SaaS solution was expanded with an additional data fetcher type that is configured to import raw sensor data received on a S3 bucket into the proNova system by the data processing pipeline.
Then automatically pushed to tde’s environment via Amazon S3, where a custom data fetcher has been developed to deploy tde’s proprietary Automatic Operations Recognition system, KPI aggregations, and QA/QC checks in real-time.
Aggregated metrics, such as drilling parameters, operational consistency, flat-time efficiencies, and equipment performance, is delivered in real-time via the proNova web-based application hosted by AWS, enabling the client’s performance and engineering teams to supplement internal analysis through tde’s REST API and a direct connector to BI applications.
