Lead Python Platform Engineer
2 days ago
Role: Lead Python Platform Engineer – Architecture & Performance
Experience: 3 to 7 Years
Job Type: Full Time Employment
Design and build a shared component library/SDK for pipelines: ingestion, parsing/OCR, extraction (RegEx now, LLM/SLM later), validation, enrichment, publishing.
standardize configuration, packaging, versioning, CI/CD, and documentation.
Create pluggable interfaces so multiple teams can swap extractors (Regex/LLM), OCR providers, and EMR publishers without code rewrites.
Define repo strategy - shared/child repos for each use case
~ Performance and reliability:
Own end-to-end profiling and tuning: cProfile/py-spy/line_profiler, memory (tracemalloc), CPU vs I/O analysis.
Instrument services with Elastic APM and correlate traces/metrics with Splunk logs; Implement robust LLM API rate limiting/governance: enforce provider TPM and concurrency caps, request queueing/token budgeting, and emit APM/Splunk metrics (throttle rate, queue depth, cost per job) with alerts.
Mentor devs, lead design reviews, codify best practices, write clear docs and examples.
Partner with ML engineers on the future LLM/SLM path (evaluation harness, safety/PII, cost/perf).
7+ years Python with strong depth in performance and concurrency (asyncio, concurrent.Splunk for logs and correlation; Must have implemented LLM based solutions and supported them in production
~ API engineering for high-throughput integrations (REST, OAuth2), resilience patterns, and secure handling of sensitive data.
~ Strong architecture/design skills: clean interfaces, packaging shared libs, versioning, CI/CD (GitHub Actions/Azure DevOps), testing.
~3+ years building large-scale data pipelines with Apache Beam and/or Spark, including hands-on Databricks experience (Jobs, Delta Lake, cluster tuning).
~ OCR (Tesseract, AWS Textract, Azure Form Recognizer), PDF parsing, text normalization.
~ LLM/SLM integration experience (e.g., OpenAI/Azure AI, local SLMs), prompt/eval frameworks, PII redaction/guardrails.
~ Cloud and tooling: AWS/Azure/GCP, Dataflow/Flink, Terraform, Docker; cost/performance tuning on Databricks.
~ Security/compliance mindset (HIPAA), secrets management, least-privilege access.
Competitive salary and benefits package
Culture focused on talent development with quarterly promotion cycles and company-sponsored higher education and certifications
Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
Annual health check-ups
Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents
Persistent Ltd. is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We support hybrid work and flexible hours to fit diverse lifestyles.
Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment
Let’s unleash your full potential at Persistent - persistent.
-
Lead Python Platform Engineer
1 week ago
Pune, Maharashtra, India Persistent Systems Full timeWe are hiring for Lead Python engineer with hands on experience with Apache Beam, databricks etc., Role: Lead Python Platform Engineer – Architecture & Performance Experience: 3 to 7 Years Job Type: Full Time Employment Design and build a shared component library/SDK for pipelines: ingestion, parsing/OCR, extraction (RegEx now, LLM/SLM later), validation,...
-
Lead Python Platform Engineer
2 days ago
Pune, India Persistent Systems Full timeAbout Position:We are hiring for Lead Python engineer with hands on experience with Apache Beam, databricks etc.,Role: Lead Python Platform Engineer – Architecture & PerformanceLocation: All PSL LocationsExperience: 3 to 7 YearsJob Type: Full Time EmploymentWhat You'll Do:Architecture and reuse:Design and build a shared component library/SDK for pipelines:...
-
Lead Python Platform Engineer
2 weeks ago
Pune, Maharashtra, India Persistent Systems Full timeAbout Position: We are hiring for Lead Python engineer with hands on experience with Apache Beam, databricks etc., Role: Lead Python Platform Engineer – Architecture & Performance Location: All PSL Locations Experience: 3 to 7 Years Job Type: Full Time Employment What You'll Do: Architecture and reuse: Design and build a shared component...
-
Lead Python Platform Engineer
1 week ago
Pune, Maharashtra, India Persistent Systems Full time ₹ 9,00,000 - ₹ 12,00,000 per yearAbout Position:We are hiring for Lead Python engineer with hands on experience with Apache Beam, databricks etc.,Role: Lead Python Platform Engineer – Architecture & PerformanceLocation: All PSL LocationsExperience: 3 to 7 YearsJob Type: Full Time EmploymentWhat You'll Do:Architecture and reuse:Design and build a shared component library/SDK for pipelines:...
-
Lead Python Platform Engineer
2 days ago
Pune, India Persistent Systems Full timeAbout Position: We are hiring for Lead Python engineer with hands on experience with Apache Beam, databricks etc., Role: Lead Python Platform Engineer – Architecture & Performance Location: All PSL Locations Experience: 3 to 7 Years Job Type: Full Time Employment What You'll Do: Architecture and reuse: Design and build a...
-
Lead Python Platform Engineer
1 day ago
Pune, India Persistent Systems Full timeAbout Position: We are hiring for Lead Python engineer with hands on experience with Apache Beam, databricks etc., Role: Lead Python Platform Engineer – Architecture & Performance Location: All PSL Locations Experience: 3 to 7 Years Job Type: Full Time Employment What You'll Do: Architecture and reuse: Design and build a...
-
Lead Python Platform Engineer
1 day ago
pune, India Persistent Systems Full timeAbout Position: We are hiring for Lead Python engineer with hands on experience with Apache Beam, databricks etc.,Role: Lead Python Platform Engineer – Architecture & PerformanceLocation: All PSL LocationsExperience: 3 to 7 YearsJob Type: Full Time EmploymentWhat You'll Do: Architecture and reuse: Design and build a shared component library/SDK for...
-
Lead Python Platform Engineer
1 day ago
Pune, India Persistent Systems Full timeAbout Position: We are hiring for Lead Python engineer with hands on experience with Apache Beam, databricks etc., Role: Lead Python Platform Engineer – Architecture & Performance Location: All PSL Locations Experience: 3 to 7 Years Job Type: Full Time Employment What You'll Do: Architecture and reuse: Design and build a shared component...
-
Lead Python Platform Engineer
2 days ago
Pune, India Persistent Systems Full timeAbout Position: We are hiring for Lead Python engineer with hands on experience with Apache Beam, databricks etc.,Role: Lead Python Platform Engineer – Architecture & PerformanceLocation: All PSL LocationsExperience: 3 to 7 YearsJob Type: Full Time EmploymentWhat You'll Do: Architecture and reuse: Design and build a shared component library/SDK for...
-
Lead Python Platform Engineer
2 days ago
Pune, India Persistent Systems Full timeAbout Position: We are hiring for Lead Python engineer with hands on experience with Apache Beam, databricks etc.,Role: Lead Python Platform Engineer – Architecture & PerformanceLocation: All PSL LocationsExperience: 3 to 7 YearsJob Type: Full Time EmploymentWhat You'll Do: Architecture and reuse: Design and build a shared component library/SDK for...
