Lead Databricks Engineer

Be in IT is a recruitment company specializing in comprehensive support for clients' IT technology needs. Operating since 2016 and as a freelancer since 2021, the company focuses on delivering precise and comprehensive job offers and effective recruitment conversations. It holds a certificate from the National Employment Agency Register (number 27663).

• Minimum 12 years of IT project experience in global and complex organizations.
• Experience in direct communication with business and senior stakeholders.
• Strong software engineering experience, more important than analytical, BI, or data science profiles.
• Knowledge of data engineering fundamentals beyond batch ETL.
• Experience with data lakehouse platforms, especially Databricks.
• Experience with streaming/near-real-time architectures, including Databricks Structured Streaming and Delta Live Tables.
• Familiarity with Power BI integration patterns.
• Knowledge of Microsoft Azure and M365 services: Functions, Service Bus or Event Hub, App Services, Cosmos DB; knowledge of MongoDB.
• Experience with DevOps/DataOps: CI/CD, cloud infrastructure management via Infrastructure as Code.
• Experience with AI development tools and agents: GitHub Copilot, Cursor, OpenAI Codex, Claude Code, Gemini CLI, Antigravity.
• Comfortable working in uncertainty and actively shaping the platform.
• English language proficiency at B2–C1 level for international communication.
• Availability ASAP preferred; acceptable maximum monthly notice period ending at month-end.

Nice to have:

• M365
• MongoDB
• Power BI

• Tech Lead / Team Lead for the Sightline platform (Databricks Lakehouse).
• Designing and developing abstraction layers and Databricks frameworks.
• Supporting teams in building data products (streaming and batch).
• Transforming legacy Data Warehouse to event-driven / streaming architectures.
• Implementing developer-first solutions beyond traditional batch ETL.
• DevOps for data platforms: CI/CD, Infrastructure as Code, operational stability.
• Monitoring, analyzing, and optimizing costs and consumption patterns in Databricks.
• Integrations with downstream tools: Power BI, Microsoft Fabric, and others.
• Mentoring engineers and defining technical standards.