Senior Python Data Scraping Engineer (Freelance)

Mindrift is an AI technology platform connecting specialists with innovative technology projects. The role is for the Tendem project, focused on developing specialized data scraping workflows for real-world applications.

• At least 5+ years of experience in data engineering, web scraping, automation, or software development.
• Strong experience in Python web scraping using BeautifulSoup, Selenium, or similar, including dynamic content (JavaScript, AJAX, infinite scroll) and APIs via proxies.
• Proven ability to extract data from complex structures, including hierarchies, archived pages, and inconsistent HTML.
• Solid background in data cleaning, normalization, and validation delivering structured datasets (CSV, JSON, Google Sheets).
• Experience handling anti-bot mechanisms and dynamic site structures at scale.
• Familiarity with cloud infrastructure such as AWS and containerization with Docker.
• Hands-on experience with LLM frameworks like LangChain and OpenRouter applied to automation tasks.
• Strong attention to detail and commitment to data accuracy.
• Self-directed work ethic with ability to troubleshoot independently.
• English proficiency at Upper-intermediate (B2) level or above.

• Own end-to-end data extraction workflows across complex websites ensuring accuracy and reliable delivery of structured datasets.
• Leverage available tools and custom workflows to speed up data collection, validation, and task execution.
• Ensure reliable extraction from dynamic and interactive web sources, adapting to JavaScript-rendered content and site changes.
• Enforce data quality standards through validation checks, cross-source consistency controls, and formatting adherence.
• Scale scraping operations for large datasets using batching or parallelization, monitor failures, and maintain stability.