1104 | Senior Data Engineer (Databricks)

Intetics Inc. is a leading American technology company providing custom software application development, distributed professional teams creation, software product quality assessment, and "all-things-digital" solutions built with SMAC, RPA, AI/ML, IoT, blockchain, and GIS/UAV/LBS technologies.

• 4+ years of data engineering experience.
• At least 2 years on Databricks or the Apache Spark ecosystem across Azure and/or AWS.
• Proficiency in PySpark, SQL, and Python with experience building production-grade pipelines under SLA constraints.
• Hands-on experience with Delta Lake schema evolution, ACID transactions, optimize/vacuum lifecycle, incremental and streaming processing.
• Hands-on experience with pipeline performance tuning and compute optimization in Databricks.
• Solid knowledge of PostgreSQL including query optimization and schema design.
• Experience with legacy ETL tooling (SSIS, Informatica, custom Python/SQL).
• Experience with large-scale multi-tenant architectures focusing on tenant isolation and data privacy.
• Proven collaboration skills across Data Science, Product, and Infrastructure teams.
• Strong understanding of data governance, security, and compliance principles.

Nice to have:

• Experience operating Databricks workspaces on both Azure and AWS.
• Experience optimizing Databricks workloads in Serverless environments.
• Experience with Microsoft SQL Server in data engineering or ETL.
• Exposure to ML feature engineering or feature stores (Databricks Feature Store, Feast).
• Experience with customer onboarding automation or Infrastructure as Code (IaC) for tenant pipelines.
• Databricks Certified Data Engineer Associate or Professional certification.

• Own Databricks production support for the predictive data platform, including monitoring, alerting, and incident response.
• Maintain and report on SLA performance metrics for data pipeline delivery.
• Identify and implement pipeline optimizations to reduce compute costs and improve throughput.
• Migrate legacy ETL/ELT pipelines to Databricks and build automation tooling.
• Support new customer onboarding by provisioning and validating tenant data pipelines.
• Design and build high-performance Databricks pipelines handling ERP and CRM data across Azure and AWS.
• Own Delta Lake architecture including schema design, partitioning, data quality enforcement, and processing patterns.
• Enforce data security best practices across Databricks environments.
• Implement data quality monitoring and observability for pipeline health and ML model inputs.
• Apply and enforce multi-tenant data isolation patterns.
• Partner with Enterprise Architecture to integrate data pipelines with product ecosystem.
• Participate in on-call rotation and after-hours incident response.
• Maintain technical documentation, runbooks, and architectural decision records.
• Apply CI/CD best practices including version control, automated testing, and deployment tooling.