QA Engineer (m/f/n)

SHIMI Sp. z o.o.

• Senior-level experience with independent QA leadership in at least one project.
• Experience creating and implementing test concepts and test strategies.
• Excellent knowledge of test automation using Python and pytest.
• Experience configuring and maintaining tests in CI pipelines.
• Experience testing backend and data systems, including API, data pipelines, batch processing, and integration.
• Experience generating synthetic or rule-based test data.
• Experience with runtime, performance, and volume testing of backend or batch systems.
• Experience testing parallel processing.
• Ability to design performance test scenarios and define metrics for bottleneck analysis.
• Experience automating frontend tests using Playwright or a comparable tool.
• Experience with manual and exploratory testing.
• Experience with defect classification, severity levels, and release criteria.
• English language proficiency at C1 level (project language).

Nice to have:

• Experience testing AI / ML / LLM systems.
• Experience with evaluation and working with nondeterministic results.
• Knowledge of Langfuse or similar tools.
• Experience creating AI-supported test agents or LLM-based test automation.
• Property-based testing.
• Experience testing optimization or planning systems.
• Experience validating PDFs or print materials.
• Experience in publishing/media industry.
• German language knowledge.

Test concept and QA approach:

• Create and maintain test concepts.
• Define test levels and responsibilities between development and QA.
• Perform risk-based prioritization.
• Define approach to test environments and test data.
• Define defect classification, severity levels, blocker criteria.
• Define acceptance and release criteria for test and production releases.
• Prepare documentation for onboarding new testers.

Test automation:

• Prepare and implement test automation concepts.
• Perform integration and API testing.
• Test data pipelines.
• Automate tests in Python using pytest.
• Integrate tests with CI pipelines.
• Perform PDF regression tests including visual comparison, preflight, and PDF/X-3 validation.
• Automate frontend tests using Playwright or similar tools.
• Optionally use AI-supported testing such as LLM-driven browser agents or test data generation.

Test data:

• Generate synthetic, diverse test data based on defined rules.
• Handle incomplete data, format errors, duplicates, and edge cases.
• Test various combinations of issue sizes and ad structures.
• Build and maintain reference datasets from real issues.
• Use property-based testing (e.g., Hypothesis).

Quality measurement:

• Define and automatically collect quality metrics.
• Monitor trends and regression.

Runtime, performance, and volume testing:

• Measure execution time, throughput, and result quality.
• Test large issues and repeated runs due to late changes.
• Test parallel production of multiple titles.
• Analyze runtime and result quality of solvers.
• Conduct volume tests based on synthetic data.
• Profile and analyze metrics in Grafana.
• Test scaling in AWS EKS.
• Consider target deployment scale and expansion plans.

Manual testing:

• Conduct exploratory testing.
• Perform release testing where automation is unfeasible or not implemented.
• Plan and document manual tests.