Architect (Data Platform)

Description We are seeking a high-impact technical leader to join our R&D organization as the Architect for our Data Platform. Reporting directly to the SVP Engineering, you will serve as the technical owner of our petabyte-scale lakehouse architecture and the surrounding infrastructure, leveraging a custom-built stack on AWS and open-source technologies. In this hands-on, individual contributor role, you will define the strategic direction for how Vi stores, processes, and serves data across analytics, classical ML, and generative AI workloads. You own the architectural bar for the platform and lead by example: authoring RFCs, writing production code and IaC, and partnering across squads to drive implementation and technical excellence. You will collaborate deeply with engineers, data scientists, business analysts, and executive stakeholders to ensure the platform meets the business's evolving needs. Responsibilities Serve as the primary technical owner for the data platform's lifecycle, encompassing ingestion, storage, modeling, and processing through to serving, governance, and cost optimization Direct the evolution of our lakehouse architecture to align with complex product requirements while ensuring robust performance at scale Architect the platform with an AI-first mindset, designing systems that power production-grade feature pipelines, model training, batch inference, and agentic workflows Partner with DevOps, who own infra execution, to architect AWS resources (Glue, EMR, EKS, networking) in CDK - getting hands-on with code and IaC where it matters most Maintain a hands-on approach by authoring production code and IaC, establishing high standards through prototypes, pull requests, and reference implementations Facilitate critical cross-squad technical decisions via RFCs and design reviews, driving engineering excellence across the organization Guarantee the non-functional integrity of the platform, specifically prioritizing reliability, performance, security, observability, and cost-efficiency Requirements 7+ years of expertise in software or data engineering, with significant experience in high-level individual contributor roles (Architect, Staff, or Principal) Extensive production-grade experience with AWS data services (S3, Glue, EMR, Athena) and the Iceberg open table format Proven proficiency in designing and operating high-scale distributed data pipelines using Python and Spark or comparable frameworks Strong hands-on experience with AWS compute services (Batch, ECS, EKS) and IaC, AWS CDK preferred Direct experience architecting production systems for AI/ML workloads, including feature pipelines, training data preparation, inference services, and vector stores Demonstrated ability to navigate architectural tradeoffs amidst technical and business constraints, communicating rationale effectively to cross-functional stakeholders Nice to have Experience with healthcare or regulated data (HIPAA, OMOP, claims, EHR) Familiarity with SageMaker, Bedrock, or other managed AI services