Kratos Growth

Senior Data Engineer (AI Consumer Intelligence Platform)

Brak informacji o wynagrodzeniu

SeniorZdalnie·New YorkFull-time·B2B

#445574·Dodano 28 dni temu·7

Źródło: justjoin.it

Aplikuj teraz

Tech Stack / Keywords

NLP PySpark Azure Kubernetes Databricks SQL Python

Podsumowanie oferty

As a Senior Data Engineer, you will architect and scale production data pipelines powering NLP and ML systems processing billions of multilingual data points daily. You will own the complete data lifecycle, establish DataOps/DevOps standards, and design governance mechanisms for scalable products. The role involves high ownership and architectural decisions impacting Fortune 500 companies' access to real-time consumer intelligence.

Wymagania

5+ years building and maintaining production ETL/ELT data pipelines
2+ years working with text/NLP data including tokenization, embeddings, and multilingual processing
3+ years of shipping data products to production serving active business users
4+ years production experience with Python (required)
2+ years with PySpark or 1+ years with Spark plus strong Python skills
3+ years with SQL including complex queries and performance optimization
1+ years production use of Databricks, including notebooks, Delta Lake, and job scheduling
2+ years experience with cloud platforms, preferably Azure
Experience deploying containerized applications to Kubernetes
Bachelor's degree in Computer Science, Data Science, Engineering, or related quantitative field

Nice to have:

Experience with Databricks Delta Lake architecture and Unity Catalog
Deep knowledge of Azure ecosystem including Data Factory, Databricks, Blob Storage, DevOps
Experience integrating LLM APIs such as OpenAI, Anthropic, or Azure OpenAI in production
Experience with LLM fine-tuning
Familiarity with observability tools like DataDog, Grafana, or Azure Monitor
Experience processing 1B+ records at scale
Multilingual text processing experience with 3+ non-English languages
Familiarity with NLP libraries such as Spacy, NLTK, CoreNLP, and TextBlob
Experience in consumer insights, CPG/FMCG, or advertising technology domains
Experience working in early-stage, high-growth companies

Obowiązki

• Design, build, and maintain production data pipelines processing 10M+ text records daily across multiple languages • Architect scalable NLP data infrastructure using PySpark, Databricks, and Azure services • Integrate Large Language Model APIs into production pipelines for text analysis and enrichment • Establish DataOps standards including CI/CD, testing frameworks, and deployment automation • Implement observability and alerting for pipeline health, data quality, and system performance • Collaborate with data scientists to productionize ML models and NLP systems • Define data governance frameworks and quality SLAs for enterprise client delivery • Mentor team members and contribute to technical hiring as the team scales

Benefity

Long-term growth opportunity
Fully remote work (4+ hour overlap with U.S. Eastern Time Zone desired)
Work with cutting-edge NLP and ML technologies at scale
High ownership and impact on platform architecture for Fortune 500 clients
Clear growth trajectory with potential for technical leadership

Inne informacje

Please be informed that the data controller is Kratos Growth (hereinafter "controller"). You have the right to request access to your personal data, their rectification, erasure or restriction of processing, the right to object to processing, as well as the right to data portability and to lodge a complaint to the supervisory authority. Personal data will be processed for the purpose of the recruitment process. Provision of data to the extent resulting from the Act of 26 June 1974 Labour Code is mandatory. In the remaining scope, providing data is voluntary. Refusal to provide mandatory data may result in the impossibility to carry out the recruitment process. The Administrator processes mandatory data on the basis of a legal obligation incumbent upon him/her, while with regard to additional data, the basis for processing is consent. Personal data will be processed until the recruitment procedure is completed and for the period of the possibility of asserting potential claims, and in the case of consent to participate in future recruitment procedures - until the withdrawal of such consent. Consent to the processing of personal data can be withdrawn at any time. The recipient of the data is the Just Join IT service and other entities to whom we have entrusted the processing of data in connection with recruitment.

Kratos Growth

10 aktywnych ofert

Zobacz wszystkie oferty

Aplikuj teraz