Link Group

Senior Webscraping & Data Engineer

38k - 48k PLN38 000 - 48 000 PLN/ mies./ mies.UoPUmowa o pracę (brutto)

SeniorHybryda·WarszawaFull-time·Umowa o pracę

#450949·Dodano miesiąc temu·5

Źródło: nofluffjobs.com

Aplikuj teraz

Tech Stack / Keywords

Data engineeringWeb ScrapingPython pandas HTML JavaScriptAPIData pipelinesAlgorithmsData structuresData qualityTroubleshootingProblem-SolvingCommunication skillsCollaboration skillsAlternative dataAirflow Kafka Docker Kubernetes AWS LLMMCPaudio/video data

Podsumowanie oferty

The Senior Webscraping & Data Engineer role focuses on building scalable systems for collecting data from diverse online sources, including websites and APIs. The position involves developing high-performance web scraping tools, cleaning and transforming data, and supporting research and investment teams with reliable datasets. The role also includes working with automation and AI-driven solutions for data extraction and validation.

Wymagania

6+ years of experience in software or data engineering.
Strong experience with web scraping, crawling and automated data collection.
Excellent Python skills, including pandas.
Strong SQL and database knowledge.
Good understanding of HTML, JavaScript, APIs and web technologies.
Knowledge of networking concepts related to web data collection.
Experience building data pipelines.
Understanding of algorithms, data structures and data quality.
Experience with large-scale or distributed data collection.
Strong troubleshooting and problem-solving skills.
Interest in AI/LLMs and agentic technologies.
Strong communication and collaboration skills.

Nice to have:

Experience with alternative data in finance or research.
Background in data-intensive organizations.
Experience building tools for researchers, analysts, quants or data scientists.
Knowledge of Airflow, Kafka, Docker or Kubernetes.
Experience with AWS or other cloud platforms.
Familiarity with LLM tools, MCPs, agents or autonomous workflows.
Experience with audio/video data.
Knowledge of statistics and data analysis.

Obowiązki

Design and develop scalable web scraping and data collection systems.
Build scrapers and crawlers for websites with different structures and technologies.
Collect data from websites, APIs and dynamic online sources.
Clean, transform and standardize raw data.
Improve accuracy, speed, coverage and reliability of scraping solutions.
Troubleshoot website changes, data quality and performance issues.
Build automated pipelines for processing collected data.
Explore AI/ML and agentic technologies for extraction and validation.
Support infrastructure for large-scale data collection.
Implement monitoring and data quality controls.
Work with researchers and analysts to understand data needs.
Explore new technologies and hard-to-source data.

Link Group

544 aktywne oferty

Zobacz wszystkie oferty

Aplikuj teraz