Senior Python Data Engineer

2 weeks ago

Thane, India iVoyant Full time

About the Project:Join a dynamic engineering team working on a high-impact tax reporting platform for the 2025 fiscal season. The core goal is to modernize and significantly accelerate the generation of Excel-based reports, reducing report generation time from minutes to seconds. This involves ingesting data from multiple upstream systems, transforming it using efficient data processing libraries, and serving it through APIs. The team emphasizes scalability, maintainability, and developer productivity, incorporating spec-driven development powered by AI tools.You’ll be contributing to the backend architecture and data pipeline powering this transformation, helping to evolve a high-performance system that's central to the client’s reporting domain.Responsibilities:Design, build, and maintain high-performance data processing pipelines using Python libraries (Pandas required; Polars nice-to-have).Develop and expose RESTful APIs using FastAPI or similar frameworks.Consume and process normalized Parquet files from multiple upstream sources to generate dynamic Excel reports.Contribute to a spec-driven development workflow (using GitHub Copilot, Claude, etc.) to scaffold and generate API/data pipeline code.Optimize report generation logic for speed and scalability, currently targeting sub-20 second response times.Integrate with messaging and storage mechanisms (e.g., Service Bus, Storage Accounts).Collaborate on infrastructure-as-code automation using Bicep similar tools (Terraform, CDK).Participate in design discussions for future migration to Snowflake and/or a data lake architecture.Contribute to CI/CD pipelines using GitHub Actions.Required Skills and Experience:Strong proficiency in Python for data processing with hands-on expertise in Pandas.Ability to quickly learn new frameworks such as Polars if needed.Experience building backend services or APIs using frameworks like FastAPI.Solid understanding of data modeling principles (Star Schema) and handling normalized datasets.Familiarity with enterprise messaging patterns and data integration from various sources (API-based and file-based).Experience working with GitHub and CI/CD pipelines (GitHub Actions or similar).Infrastructure-as-Code experience with Bicep or comparable tools (Terraform, AWS CDK).Comfort with spec-driven development and leveraging AI tools like GitHub Copilot for scaffolding.Nice-to-Have / Preferred Qualifications:Experience with Polars (not required).Experience with orchestration tools like Apache Airflow or ETL frameworks like DBT.Exposure to Snowflake (streams, tasks, stored procedures).Experience working with DuckDB and/or DocTV/OpenTI XLarge for report generation.Knowledge of Angular or frontend plugins for Excel is a plus but not required.Familiarity with async workflows and distributed processing concepts.Soft Skills & Cultural Fit:Strong ownership mindset, proactive in identifying and solving problems.Collaborative team player, able to work cross-functionally with architects, engineers, and stakeholders.Comfortable working in a distributed, multicultural team across multiple time zones (Europe, India, U.S.).Adaptable and open to learning emerging tools and technologies.Work Environment / Time Zone Expectations:Global team with flexibility on candidate location.Current team members are distributed across the US, Latin America, Europe, and India.Candidates from other regions will be considered, provided there is reasonable overlap with core collaboration hours.

  • Senior data engineer

    2 days ago

    Thane, India K&K Talents Full time

    K&K Talents is an international recruiting agency that has been providing technical resources globally since 1993. This position is with one of our clients in India , who is actively hiring candidates to expand their teams. Role : Senior Data Engineer (Tableau + Real Estate Domain) Locations: Pune | Bangalore | Hyderabad | Gurgaon | Chennai | Noida | Mumbai...

  • Senior Data Engineer

    3 weeks ago

    Thane, India Egen Full time

    Senior Data Engineer – GCPJob Overview: We are looking for a skilled and motivated Senior Data Engineer with strong experience in Python programming and Google Cloud Platform (GCP) to join our data engineering team. The ideal candidate will be responsible for designing, developing, and maintaining robust and scalable ETL (Extract, Transform, Load) data...

  • Senior Python Developer

    2 weeks ago

    Thane, India Antal International Full time

    About the Role We are building cutting-edge AI products designed for enterprise-scale applications and are looking for a Senior Python Developer to join our core engineering team. You will be responsible for designing and delivering robust, scalable backend systems that power our advanced AI solutions. Key Responsibilities • Design, develop, and maintain...

  • Senior Data Engineer

    3 weeks ago

    Thane, India Atomic North Full time

    Position:Senior Data Engineer (Azure) Experience:6+ years Skills:Azure Databricks, PySpark, Python, SQL, Azure Data Factory, Data Modelling, Data Migration, Data Warehousing Location:India (Remote / Hybrid options available) Interview Mode:Virtual Notice Period:Immediate to 30 DaysKey Responsibilities Design and developdata pipelinesandETL...

  • Senior Full Stack Engineer

    6 days ago

    Thane, India Waveonline LLC Full time

    WE’RE HIRING – Senior Full Stack Engineer (Python + Angular)Kochi, Kerala | On-siteWave Online Infoway(p) LtdExperience: 4+ YearsWave Online is expanding our core engineering team, and we’re looking for a Senior Full Stack Engineer who can take full ownership of both backend and frontend systems. This is a high-impact role where you’ll help architect...

  • Senior AI Engineer

    7 days ago

    Thane, India Qode.world Jobs Full time

    Job Description : We are looking for a Senior AI Developer to join our team at Thane (Maharashtra India). We are seeking someone with 6-8 years of professional experience in designing, training, and deploying advanced AI/ML models across domains such as computer vision, natural language processing, and generative AI. Expertise in Python and...

  • Senior Engineer, Data Engineering

    7 days ago

    Thane, India Envu Full time

    Data Engineer TYPE: Permanent We are seeking a Data Engineer with strong analytical and technical expertise to support our data-driven transformation. You will be responsible for building and maintaining data infrastructure and pipelines, enabling robust analytics and machine learning initiatives across multiple departments This role requires hands-on...

  • Data Engineer

    7 days ago

    Thane, India Envu Full time

    We’re Hiring! Data Engineer At Envu, we partner with our customers to design world-class, forward-thinking innovations that protect and enhance the health of environments around the world. We offer dedicated services in: Professional Pest Management, Forestry, Ornamentals, Golf, Industrial Vegetation Management, Lawn & Landscape, Mosquito Management, and...

  • Data Engineer

    1 week ago

    Thane, India Envu Full time

    We’re Hiring! Data Engineer At Envu, we partner with our customers to design world-class, forward-thinking innovations that protect and enhance the health of environments around the world. We offer dedicated services in: Professional Pest Management, Forestry, Ornamentals, Golf, Industrial Vegetation Management, Lawn & Landscape, Mosquito Management, and...

  • Data Engineer

    3 weeks ago

    Thane, India Envu Full time

    We’re Hiring! Data Engineer At Envu, we partner with our customers to design world-class, forward-thinking innovations that protect and enhance the health of environments around the world. We offer dedicated services in: Professional Pest Management, Forestry, Ornamentals, Golf, Industrial Vegetation Management, Lawn & Landscape, Mosquito Management, and...