Senior Data Engineer

The role is within the Encarta squad focused on the Personalization mission. The squad owns User Behavior Knowledge (UBK), the platform that powers how users listen to, watch, and care about content, directly impacting music personalization systems. The project involves migrating legacy batch affinity datasets to a modern event-driven stack and improving cost efficiency and maintainability in GCP.

• Experienced software engineer with a strong background in data engineering and ability to independently own end-to-end deprecation and migration work.
• Proficiency in Scala and Java, with experience in data pipelines and batch processing frameworks.
• Comfortable working with SQL and data analytics platforms such as BigQuery, including auditing endpoint usage and managing dataset lifecycles.
• Experience with GCP infrastructure including cloud storage, compute, and cost management.
• Proven track record in safely migrating or deprecating production systems, auditing downstream consumers, coordinating stakeholders, and decommissioning infrastructure without disruption.
• Ability to navigate complex codebases and responsibly tear down existing systems.
• Autonomous working style with ability to meet clear scopes and milestones with minimal guidance.
• Commitment to quality and reliability, with methodical, well-documented approaches.
• Proactive communication skills, especially in coordinating migration timelines.
• Experience working in agile environments with continuous delivery, automated testing, and defensive programming.

• Audit and migrate downstream consumers of 16+ legacy batch music affinity endpoints to existing Mirage view exports.
• Deprecate legacy audiobook and podcast affinity and listening history endpoints, replacing them with new system equivalents and confirming data parity.
• Tear down the Dawnshard system once upstream dependencies are deprecated.
• Coordinate with downstream teams to validate migrations and ensure smooth transitions.
• Decommission legacy BigQuery endpoints and the descriptor-affinity pipeline to reduce maintenance burden and incident risk.
• Progressively unlock cost savings throughout the engagement, targeting an initial $60K/month reduction within two months.

Workplace is remote within Sweden.