Overview
A leading U.S.-based animal health company partnered with Excelra to address the challenge of fragmented microbiome data spread across public and internal sources. Excelra developed a scalable microbiome data platform featuring a centralized PostgreSQL database, standardized metadata, and an interactive R Shiny dashboard. This unified solution enabled seamless querying, visualization, and management of over 52,000 curated studies and 1.1 million samples. The platform improved research efficiency by reducing manual curation efforts by 65% and empowering teams to explore complex datasets with ease. The result: accelerated veterinary research, better cross-study insights, and a future-ready infrastructure for ongoing microbiome innovation.
Our client
Our client is a leading U.S.-based animal health company that leverages data-driven research and bioinformatics to advance veterinary science. Their R&D teams extensively use microbiome data to study disease mechanisms and animal health. To streamline research, they needed a consolidated microbiome data repository with an interactive dashboard that enables efficient exploration, filtering, and analysis.
Client’s challenge
A U.S. based Computational Sciences Group of a leading global animal health company aimed to unlock value from microbiome data dispersed across public repositories and internal databases. The challenge was to identify, screen, curate, validate, standardize and integrate diverse microbiome datasets, enabling seamless search, analysis, and visualization.
Client’s goals
Since there was no single data source that has all the microbiome-related data, exploring as well as analysing large volumes of data without a proper user interface was a tedious task. Client wanted to have a centralised microbiome-related database and a responsive user interface to navigate through the data easily.
Our approach
Excelra developed a scalable, integrated bioinformatics platform to aggregate, standardize, and visualize microbiome data, supporting advanced research workflows.
Data acquisition & integration
- Aggregated microbiome datasets from public repositories like HMP, JGI, MG-RAST, Disbiome, and GMrepo.
- Merged public datasets with internal sample data to provide a unified view.
- Stored all data in a centralized PostgreSQL database.
Data curation & normalization
- Developed SOPs, curation strategies, and database schema for consistent data integration.
- Normalized metadata using reference ontologies to harmonize variables such as sample type and taxonomy.
- Enabled cross-study analysis by standardizing naming conventions and data structures.
Data infrastructure
- Organized curated data into relational formats for faster querying and efficient storage. PostgreSQL was used for reliable, high-performance management of structured biological data.
Dashboard & visualization
- Built an interactive dashboard using R Shiny Server.
- Enabled users to query, filter, and visualize datasets through an intuitive interface.
- Added functionality for users to upload, download, and manage data efficiently.
Our solution
Excelra delivered an end-to-end microbiome data management platform designed to drive discovery and insight. Key components included:
- Centralized Data Infrastructure: A PostgreSQL database housing over 52,000 curated studies, 1.1 million samples, and 26,534 raw data files—normalized and standardized for consistency and quality.
- Interactive Analytics Dashboard: A Shiny-based interface enabling advanced querying, dynamic filtering, and intuitive data visualization to facilitate deep exploration of complex datasets.
- Integrated Data Ecosystem: Seamless unification of public and proprietary microbiome data into a single, searchable platform.
- Scalable Architecture: A flexible, extensible system built to support future data growth and evolving analytical needs.
By consolidating fragmented data sources into a cohesive, accessible platform, Excelra’s solution empowers research teams to generate actionable insights faster—accelerating discovery, optimizing R&D workflows, and unlocking the full potential of microbiome data.
Conclusion
The integration of over 52,000 curated studies and 1.1 million microbiome samples significantly improved the client’s ability to explore and utilize microbiome data, reducing manual effort in data curation and wrangling by 65%. Researchers could now access millions of data points through a centralized, searchable interface empowering faster discovery, cross-study comparisons, and actionable insights in animal health research.
