Senior Python Data Scraping Engineer (Freelance)

Mindrift is a platform connecting specialists with AI technology projects, focusing on developing high-quality AI technologies by combining professional expertise with advanced AI development.

• At least 5+ years of experience in data engineering, web scraping, automation, or software development.
• Strong experience in Python web scraping using tools like BeautifulSoup, Selenium, including handling dynamic content (JS, AJAX, infinite scroll) and APIs via proxies.
• Proven ability to extract data from complex and inconsistent web structures.
• Solid skills in data cleaning, normalization, validation, and delivering structured formats such as CSV, JSON, and Google Sheets.
• Experience handling anti-bot mechanisms and dynamic site structures at scale.
• Familiarity with cloud infrastructure (AWS or equivalent) and containerization (Docker).
• Hands-on experience with LLM frameworks (LangChain, OpenRouter) applied to automation.
• Strong attention to detail and independent troubleshooting skills.
• English proficiency at Upper-intermediate (B2) or above.

• Own end-to-end data extraction workflows across complex websites, ensuring coverage, accuracy, and reliable structured dataset delivery.
• Leverage tools and custom workflows to accelerate data collection, validation, and task execution.
• Extract reliably from dynamic and interactive web sources, handling JavaScript-rendered content and changing site behaviors.
• Enforce data quality standards via validation checks, consistency controls, formatting adherence, and systematic verification.
• Scale scraping operations for large datasets using batching or parallelization; monitor failures and maintain stability against site changes.

Work is remote but includes connection to a project based in Wrocław, Poland. English proficiency required at minimum B2 level.