Senior Python Data Scraping Engineer (Freelance)

Mindrift is connected to the Tendem project which develops AI technologies by combining expert knowledge with AI development.

• At least 5+ years of experience in data engineering, web scraping, automation, or software development.
• Bachelor’s or Master’s degree in Engineering, Applied Mathematics, Computer Science, or related fields is a plus.
• Strong experience with Python web scraping including libraries like BeautifulSoup, Selenium or similar.
• Ability to work with dynamic content such as JS, AJAX, infinite scroll and APIs via proxies.
• Experience extracting data from complex structures including hierarchies, archived pages, and inconsistent HTML.
• Background in data cleaning, normalization, and validation delivering structured datasets (CSV, JSON, Google Sheets).
• Experience handling anti-bot mechanisms and dynamic site structures at scale.
• Familiarity with cloud infrastructure like AWS and containerization via Docker.
• Hands-on experience with LLM frameworks like LangChain, OpenRouter for automation.
• Strong attention to detail and independent troubleshooting skills.
• English proficiency at Upper-intermediate (B2) or above.

• Own end-to-end data extraction workflows across complex websites ensuring accuracy and structured dataset delivery.
• Leverage tools and custom workflows for data collection, validation, and task execution.
• Extract data reliably from dynamic and interactive web sources, handling JavaScript-rendered content.
• Enforce data quality standards with validation checks and consistency controls.
• Scale scraping operations for large datasets using batching or parallelization while monitoring failures.