Senior Python Data Scraping Engineer (Freelance)

Mindrift operates the Tendem project, combining AI and human expertise to deliver specialized data scraping workflows.

• Minimum 5+ years of experience in data engineering, web scraping, automation, or software development.
• Strong experience in Python web scraping using libraries such as BeautifulSoup, Selenium, including handling dynamic content (JS, AJAX, infinite scroll) and APIs via proxies.
• Proven ability to extract data from complex and inconsistent HTML structures.
• Solid skills in data cleaning, normalization, validation, and delivering structured datasets (CSV, JSON, Google Sheets).
• Experience with anti-bot mechanisms and dynamic site structures at scale.
• Familiarity with cloud infrastructure (AWS or equivalent) and containerization (Docker).
• Hands-on experience with LLM frameworks like LangChain, OpenRouter or similar.
• Strong English proficiency at upper-intermediate (B2) level or above.
• Methodical, detail-oriented, and able to work independently.

Nice to have:

• Bachelor’s or Master’s degree in Engineering, Applied Mathematics, Computer Science, or related fields.
• Link to GitHub portfolio.

• Own end-to-end data extraction workflows across complex websites with complete coverage and reliable delivery.
• Use internal tools (Apify, OpenRouter) and custom workflows to accelerate data collection and validation.
• Adapt methods for reliable extraction from dynamic web sources including JavaScript-rendered content.
• Enforce data quality standards through validation, consistency checks, and formatting adherence.
• Scale scraping operations efficiently with batching or parallelization while monitoring failures and site changes.