How Ruten Neuro Scaled GPU-Based Research Workflows Beyond Local Machines

Business Challenge

Ruten Neuro faced critical scalability and infrastructure bottlenecks due to its reliance on a localized, GPU-based data processing pipeline running on individual local development machines (Windows, Mac, or Ubuntu). While these Python-based workflows were functional for isolated testing, the monolithic, local setup was entirely incapable of executing parallel batch processing or handling simultaneous, independent data sessions. This restriction severely throttled data processing velocity and limited the client’s ability to scale their advanced behavioral and neuroscience research workflows efficiently.

Additionally, the local architecture lacked centralized operational monitoring, making it highly difficult to accurately track processing run times, audit system performance, or diagnose specific software failure modes during multi-step computations. To unlock the next phase of scientific and operational scaling, Ruten Neuro needed to modernize its infrastructure by safely porting its Level 1 Inference and Level 2 Video Data Processing workloads to a managed cloud environment without compromising output precision.

Solution Provided

D3V Technology Solutions partnered with Ruten Neuro to architect and deploy a custom Proof of Concept (PoC) infrastructure on Google Cloud Platform (GCP), successfully modernizing their local Python-based data workflows into a high-performance, managed cloud environment.

The core implementation included:

• GPU-Accelerated Inference (Level 1): Provisioned high-performance Google Compute Engine (GCE) virtual machines equipped with high-end GPUs and custom pre-installed Python environments utilizing SLEAP via pip to automate batch inference on raw MP4 video files and safely output 2D tracks (H5 files).
• Orchestrated Video Data Processing (Level 2): Engineered a fully automated serverless orchestration using Cloud Workflows to manage the four sequential steps of the client’s core data script: Load & Validate Inputs, Clean 2D Tracks, Triangulate 3D + QC, and Compute Gape + Classification.
• High-Scale Parallel Compute Architecture: Integrated Google Cloud Batch optimized for high-scale, parallel neuroscience workloads and Nextflow integration, enabling the underlying infrastructure to scale elastically and process multiple independent sessions simultaneously.
• Enhanced Observability & Cloud Persistence: Deployed built-in metric tracking and interactive execution graphs via Cloud Workflows to give engineers complete visibility into status, run times, and localized failure modes, while persisting all consolidated final H5 files, CSVs, and analytical plots securely within Google Cloud Storage (GCS).

Tech Stack Used

• Orchestration: Cloud Workflows
• Compute (Batch): Google Cloud Batch
• Compute (Inference): Compute Engine (GPU VMs)
• Compute (Web/App): Cloud Run
• Backend & Libraries: Python (with native support for SLEAP, LFP algorithms, and W&B APIs)
• Data Persistence: Google Cloud Storage (GCS)

Project Category

Application Modernization, Cloud Migration, and High-Performance Compute Automation.

Measurable Impact / Outcome

• Massive Session Scalability: Unlocked the operational capacity to execute parallel jobs and validate infrastructure performance by processing up to 5 completely independent data sessions simultaneously, breaking the single-session bottleneck of local machines.
• Rapid Processing Turnaround: Met strict performance goals by compressing the processing time of a full session’s worth of video files (typically 5 to 10 trials) down to a predictable 1-to-3-hour timeframe.
• 100% Scientific Data Integrity: Achieved complete parity with local baselines during testing, ensuring that cloud-generated H5 files, CSVs, and scientific plots perfectly matched legacy outputs with zero data distortion.
• Granular Operational Observability: Replaced blind local execution with out-of-the-box monitoring and detailed execution graphs, allowing immediate tracking of system health, processing times, and structural failure modes.
• Cost-Efficient Cloud Onboarding: Delivered a highly scalable enterprise architecture for a remaining client balance of just $900 by successfully leveraging a $2,500 Google DAF/PSF fixed-price co-investment.

Client Testimonial

“By collaborating with D3V Technology Solutions, Ruten Neuro successfully transitioned our localized GPU data workflows into a high-performance, automated cloud environment on GCP. This infrastructure modernization allows us to scale parallel sessions seamlessly and has drastically accelerated our scientific data pipeline’s processing velocity. D3V has proven to be an exceptional partner in bringing our advanced workloads to the cloud.” — Hayato Nakamura, Co-founder & CTO, Ruten Neuro