Data Engineer
Brak informacji o wynagrodzeniu
MidFull-time
#413633·Dodano dziś·1
Źródło: EagerDevsTech Stack / Keywords
DatabricksSQLApache SparkETLAWSAzureGCPPython
Wymagania
- Minimum 3 years of experience as a Data Engineer or similar role.
- Very strong knowledge of SQL and experience designing complex data transformations.
- Experience with Databricks platform and Apache Spark.
- Knowledge of views and materialized views concepts in data warehouses.
- Practical knowledge of ETL tools and data modeling principles.
- Experience integrating data from various structured and unstructured sources.
- Ability to write optimal and efficient code for large data volumes.
- Very good command of English (working in an international environment).
Nice to have:
- Experience with cloud technologies (AWS, Azure, GCP).
- Knowledge of Python and/or Scala in the context of data processing.
- Experience with code versioning tools (e.g., Git).
Obowiązki
- Designing and developing data pipelines on the Databricks platform.
- Implementing data transformations using SQL (views, materialized views).
- Creating and optimizing Apache Spark notebooks.
- Ensuring high performance and scalability of data processing.
- Integrating various data sources, creating data models, and ensuring compliance with business requirements.
- Monitoring, maintaining, and optimizing existing ETL solutions.
- Collaborating with analytical and business teams to understand data needs.
EagerDevs
31 aktywnych ofert