Business Area:
Engineering
Seniority Level:
Mid-Senior level
At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we're the preferred data partner for the top companies in almost every industry. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises.
About Octopai
Octopai is a fast-growing Data Intelligence company (Acquired by Cloudera) that empowers enterprises to understand, trust, and optimize their data. Our platform provides automated data lineage, cataloging, and discovery to accelerate analytics, improve governance, and drive business impact.
As we expand from SaaS into large-scale enterprise and on-premise deployments, we are investing in strengthening our core platform infrastructure and runtime services — the foundational layer that powers Octopai’s ingestion, analysis, metadata processing, and operational execution. We are looking for a Staff Software Engineer to help build and evolve these critical platform components.
Role Overview
As a Staff Software Engineer (Core Platform, Runtime Services), you will focus on designing and developing the foundational runtime and shared services that enable Octopai’s platform to operate at scale.
This role is deeply technical, centered around building robust infrastructure primitives, runtime orchestration, platform-wide reliability, and performance improvements.
You will work on the systems that sit at the heart of Octopai’s execution engine — supporting metadata harvesting, lineage computation, distributed processing, and enterprise workloads across SaaS and customer-managed deployments.
What you'll do
Design and implement core runtime services that power Octopai’s platform execution, analysis workflows, and operational stability.
Build shared infrastructure components used across the backend organization, including service frameworks, job execution layers, scheduling, and platform abstractions.
Improve scalability, reliability, and performance of platform services supporting metadata ingestion and lineage computation.
Develop platform capabilities for workload orchestration, lifecycle management, retries, fault handling, and resource optimization.
Partner with DevOps and Cloud teams to ensure runtime services are deployable and manageable in Kubernetes-based and enterprise environments.
Build foundations for observability, including logging, metrics, tracing, and troubleshooting tools for core services.
Ensure runtime infrastructure supports enterprise-grade requirements such as isolation, security, compliance, and predictable operations.
Contribute to architecture discussions and evolve Octopai’s core platform foundations through strong design and implementation practices.
Support engineering quality through clean architecture, code reviews, and scalable backend patterns.
Requirements
8+ years of backend or platform engineering experience, with demonstrated impact at Staff level.
Strong expertise in building scalable backend infrastructure, runtime systems, or distributed service platforms.
Excellent programming skills in C# (.NET), Java and modern backend engineering practices.
Experience designing shared services that support orchestration, execution pipelines, or platform-level abstractions.
Hands-on experience with SQL and NoSQL databases (SQL Server, PostgreSQL, ArangoDB, or similar).
Familiarity with messaging, caching, and coordination systems (Kafka, Redis, etc.).
Experience operating containerized services and supporting Kubernetes-based deployments.
Strong understanding of production reliability, fault tolerance, observability, and operational best practices.
Ability to work across teams and collaborate effectively with engineering, infrastructure, and product stakeholders.
You may also have:
Experience with workflow engines, distributed job execution systems, or large-scale runtime platforms.