The Data Import API

Introduction

The Data Import API is used to ingest large amounts of business activity data into the SINAI platform in an automated, ongoing basis. Using your system of choice, your IT team exposes the relevant data and our API imports it automatically on a periodic basis. This data is then transformed into greenhouse gas (GHG) emissions, which can be used for sustainability analysis and to develop a decarbonization roadmap.

This automated process is a good next step for customers that have well established dataset preparation and manual upload workflows and are looking to streamline the data collection process.

The Data Import API supports uploading more than 10,000 records per batch.

Behind the scenes, the import system utilizes the same smart data processing infrastructure as the manual data upload process, with all uploaded records showing under the Data Management section of the platform.

Preparing the data for automation

Computing your GHG inventory requires a specific subset of your business data, which is then mapped to GHG emissions via a comprehensive set of formulas in the SINAI platform. This initial analysis of what data is required and where in your business it can be best sourced from is first completed as part of your account implementation phase. Our Climate Experts work side-by-side with your sustainability team to identify suitable sources of data and to establish adequate data pipelines from your organization to our system.

While not required, we have found that the most effective workflow is to first validate the sourcing and processing of data for each emissions source using our bulk upload functionality through the SINAI platform web user interface. Our system provides customer-specific Excel data templates that can be used to guide the preparation of data.

Once the flow and curation of data is well understood, it is time to automate the process using the Data Import API. Often times, the same data preparation process and IT infrastructure is used on the customer side, but the last step of exporting and uploading the data on a regular basis is replaced by our fully automated integration.

Secure and scalable imports

The Data Import API leverages the Fivetran platform for the initial data extraction step. Fivetran offers native connectors for a wide range of relational and data warehouse databases. It also supports connecting to file exchange systems solutions like sFTP and AWS S3. These are pull-based connectors, supporting full and incremental syncs, auto-detecting schema changes, and offering private connectivity options such as SSH tunnels, VPNs, and PrivateLink.

Key supported platforms include:

• On‑premises and cloud DBs: MySQL, PostgreSQL, Oracle, SQL Server
• Managed DB services: Amazon RDS/Aurora, Azure SQL, Google Cloud SQL
• Analytics-ready data warehouses: BigQuery, SAP HANA, Snowflake, Databricks, etc.
• Cloud file storage: S3, Azure Blob, Google Cloud Storage, Wasabi
• Collaborative platforms: Google Sheets, spreadsheets via file connectors

Fivetran maintains a robust, enterprise-grade security posture designed to protect customer data throughout its lifecycle. With encryption at every layer and comprehensive compliance certifications, Fivetran ensures secure, governed and resilient data movement across hybrid and multi-cloud environments:

• Encryption & Key Management: Data, credentials and keys are encrypted in transit and at rest.
• Secure Connectivity Options: Flexible secure connection modes include SSH tunnels, VPN, private networking via AWS PrivateLink/Azure Private Link/GCP Private Service Connect, on‑prem/hybrid deployments, and self-hosted HVR agents—eliminating open inbound firewall risks
• Standards & Certifications: Fivetran is HITRUST i1 certified (Gold standard for HIPAA), and maintains SOC 1 & SOC 2, ISO 27001, PCI DSS Level 1, GDPR/CCPA compliance, Cyber Essentials, and more.

Incremental integrations

While we strive to support the automation of imports of 100% of your relevant business activity data, in reality the process of data collection and curation is much more incremental. For this reason our platform has been designed to support multiple methods of data collection including individual manual record creation, bulk uploads or fully automated periodic data imports.

Configuration

Dataset connection configuration

Once you are ready to automate the import of data into the SINAI platform, our Integrations team will work with your IT team to setup the flow of data:

1. Data to be imported is being exported regularly from your IT systems
2. SINAI's IT team sends you a connection request.
3. Your IT team receives a secure connection request to capture the connection details including credentials
   1. Your credentials are securely stored encrypted in transit and at rest within the Fivetran network
4. SINAI's IT team activates the integration and verifies data is flowing properly
5. The Data Import API starts to pick up data updates automatically.

The connection configuration is typically done in a coordinated session with our your IT team and our Integrations team.

Configuring dataset exports

While the specific technology used to prepare datasets depends on your organization's unique digital architecture, exporting structured data from enterprise ERP systems and data platforms typically involves using built-in reporting tools, custom SQL queries, or API-driven data extracts to retrieve curated subsets of data:

• Systems like SAP ERP or Oracle E-Business Suite support exporting data through custom reports, IDocs, or batch jobs that output flat files in CSV or XML formats.
• In NetSuite, saved searches and SuiteAnalytics provide a user-friendly interface for querying and exporting structured data.
• Cloud data warehouses like Snowflake, BigQuery, or Databricks allow curated exports via parameterized SQL scripts or materialized views.

Our Integrations team can help in identifying the simplest solution for your organization while maintaining a high degree of flexibility to meet your unique security and technology constraints.

As indicated above, the datasets can be made available directly from the data platform of choice or through a file exchange solution like sFTP or S3.

Automated data flow

Once a connection is configured, the import process is fully automated. The SINAI Data Import API automatically detects when new datasets are published, imports them and computes the corresponding GHG emissions. Just as for the manual upload process, all imported business activity records are visible from the Data Management section in the platform and can inspected, adjusted or approved.

If the a specific subset of data fails to import due to a data or misconfiguration issue, our Integrations team will first attempt to heal the issue behind the scenes. In the event the issue cannot be resolved without your IT team's assistance, we will reach out to find a solution and restart the automated data processing flow.