Job description
Overview:
This role is intended for an M.Sc. student with hands-on industry or other experience who will focus on maintaining and operating production-grade data pipelines. The position is centered on Databricks-based data workflows, scripting, and reliable data delivery, serving as a key enabler for the AI team’s ongoing development and research needs.
The role focuses on practical, hands-on data operations to ensure reliability and consistency across AI workflows, working closely with the data infrastructure engineer responsible for data architecture, governance, and platform-level decisions within an established environment.
Key Responsibilities:
· Maintain and extend Databricks-based pipelines for ingesting, transforming, validating, and versioning structured and semi-structured data.
· Develop Python scripts and notebooks for data extraction, processing, automation, and ad?hoc dataset preparation for AI use cases.
· Manage datasets across Azure data platforms (ADLS Gen2, Blob Storage, Delta tables), including storage, access, and historical reprocessing to handle schema changes.
· Implement data quality checks, logging, and lightweight monitoring to ensure stable, traceable production data flows.
דרישות:
· M.Sc. student in Computer Science, Data Engineering, Information Systems, Biomedical Engineering, or related field, with at least one year remaining, and 1+ year of industry or military experience working with real-world or production data systems.
· Strong Python proficiency for maintainable data processing and automation, with solid SQL skills (joins, performance, analytical queries) and hands-on Git experience in shared codebases.
· Experience with Databricks or Spark-based environments (notebooks, scheduled jobs) and cloud data platforms (Azure or AWS), including managed storage and reliable, reproducible data pipelines.
· Familiarity with time-series or signal-based data (advantage for physiological/cardiac signals such as ECG).
· Independent problem solver who works well in a collaborative environment with data
infrastructure and AI teams.
ניסיון נדרש: ללא נסיון
Is this role relevant for you?