AWS BigData Architect (Databricks)

SoftServe is a global digital solutions company with headquarters in Austin, Texas, founded in 1993. The Big Data and Analytics Center of Excellence is a data consulting and engineering branch with customers in healthcare, finance, manufacturing, retail, and energy. SoftServe holds top-level partnership statuses with major cloud providers including AWS, GCP, Microsoft, Databricks, Snowflake, and Confluent.

• Specialization in data pipeline creation
• Experience with batch and streaming data processing
• Leadership skills to manage a team of data engineers
• Skilled in scalable data solutions on AWS with focus on Databricks
• Hands-on experience with Databricks on AWS including Delta Lake, Unity Catalog, Workflows, and Jobs
• Proficient in Python, Scala, or Java, preferably Python, and SQL for data querying
• Knowledgeable of big data technologies such as Apache Spark or Flink
• Familiarity with orchestration tools like Databricks Workflows, Apache Airflow, or Managed Apache Airflow (MWAA)
• Experience with data streaming platforms like Apache Kafka, Amazon Managed Streaming for Apache Kafka (MSK), or Kinesis
• Familiar with concepts of Kafka, streaming, Avro file format, GitHub, and Snowflake
• Proficient in utilizing data warehouses like Amazon Redshift or Snowflake
• Ability to translate customer requirements into development tasks, estimate timelines, and guide teams

• Own source-to-target mappings and coordinate with business stakeholders on data ingestion
• Assist the Data Engineering Team with implementation scopes
• Participate in architecture decisions and scale data platforms by planning ingestion pipelines and maintaining data models
• Design and implement Databricks-based data solutions within AWS environments
• Drive requirements gathering with business and tech stakeholders
• Assist with data modeling for systems using Snowflake and Kafka
• Create and maintain documentation for data schema and models
• Work in a team dedicated to continuous learning, improvement, and knowledge-sharing
• Be involved in the full project lifecycle from design, proofs of concept to MVP and full-scale implementation
• Investigate new technologies, build prototypes, and share knowledge within the SoftServe Big Data Community