Web Scraping Engineer

Oxylabs Web Intelligence team builds and delivers large-scale, custom web scraping datasets for clients.

• Multiple years of backend development experience with Python or Go, or strong web scraping expertise driven by AI tools.
• Familiarity with web scraping fundamentals, parsing, headless browsers, and databases such as PostgreSQL or BigQuery.
• High level of independence and ownership in delivery.
• Business-first mentality prioritizing speed and delivery over perfection.
• Hands-on experience and comfort with AI-driven development tools (e.g., Cursor, Claude Code, Antigravity).

Nice to have:

• Experience in data engineering including ETL/ELT pipelines, Spark, Airflow, Dagster.

• Design and build high-scale crawlers, scrapers, and supporting infrastructure handling thousands to millions of requests.
• Own projects end-to-end from client requirements to delivery, building custom solutions without waiting for detailed specs.
• Write functional, reusable code primarily in Python or Go, leveraging AI-driven development tools.
• Work within the data platform to enable seamless building and delivery of large datasets.
• Use internal scraping tools or build new tools as client priorities evolve.
• Justify technical decisions, perform code reviews, and collaborate closely with the team.