Data Engineer

3 hours ago

Pune Maharashtra India, Maharashtra Solytics Partners Full time

Company Profile: Solytics Partners is a Global Analytics firm, recognized with multiple industry awards for innovation and excellence. Our team comprises experts with deep knowledge in risk, analytics, AI/ML, AML/FCC, and fraud. By converging this expertise with cutting edge technologies like AI, Machine Learning, Generative AI, and Large Language Models (LLMs), we deliver powerful automated platforms and incisive point solutions. Our offerings enable clients to streamline and future-proof their risk, AML, and analytics processes, comply seamlessly with global regulations, and safeguard financial systems. Whether it’s solving complex challenges or driving operational efficiency, Solytics Partners is committed to empowering organizations with transformative tools to stay ahead in an evolving regulatory landscape.Job Title: Data Engineer (Web Scraping)Experience: 5 – 10 years of relevant experienceLocation & Timings: Pune – Work from office & Timing - 11:00 AM – 8:00 PMEducation Qualification: Masters or bachelor's in computer science or IT or in other relevant discipline from a reputed institute.Role Type: Permanent / Full TimeJob Description: We are seeking an experienced Data Engineering & Automation Lead to design, automate, and optimize large-scale data processing and web scraping pipelines. The role involves leading a team to build and maintain high-performance ETL workflows using Apache Airflow, Apache Spark, and AWS services, while integrating AI/NLP solutions powered by OpenAI GPT and other GenAI models for intelligent data extraction and analytics.Responsibilities: Design, automate, and maintain ETL and data processing pipelines using Apache Airflow and Apache Spark. Build, monitor, and optimize web scraping and data extraction workflows for global compliance and risk data sources. Lead and manage web scraping and data engineering teams, ensuring delivery excellence, code quality, and scalability. Create, design, and document automation workflows and secure data-sharing systems using AWS (Lambda, S3, API Gateway, SQS). Implement AI and NLP integrations using OpenAI GPT and GenAI models for intelligent data extraction, tagging, and analytical automation. Analyze large-scale datasets to identify quality gaps, improve accuracy, and optimize indexing and retrieval performance. Collaborate with Backend, DevOps, and Frontend teams for data modeling, monitoring, and visualization. Work closely with clients to gather and translate business requirements into scalable automation and analytics solutions.Author HLD/LLD documentation, mentor junior engineers, and continuously improve automation processes and data workflows. Required Skills: Programming: Python, SQL, JavaScript Data Engineering & Automation: Apache Airflow, Apache Spark, Web Scraping (Scrapy, Selenium), Pandas, NumPy Databases & Storage: Elasticsearch, MongoDB, MySQL Cloud & Backend: AWS (Lambda, S3, EC2, CloudWatch, SQS, SNS, EKS), Docker, Django, Flask AI/ML & NLP: OpenAI GPT APIs, NER, Sentiment Analysis, Embeddings, Information Extraction Monitoring & Tools: Grafana, Git, Postman, Jupyter, VS Code Good to Have Strong understanding of Large Language Models (LLMs) and Generative AI for building intelligent data extraction and analytics agents. Familiarity with risk and compliance domains, including Sanctions, PEP (Politically Exposed Persons), and AMS (Adverse Media Screening) data and processes. Soft Skills: Leadership & Team Mentoring Problem-Solving & Analytical Thinking Clear Technical Communication Cross-functional Collaboration

  • Data Privacy Consultant

    3 hours ago

    Pune, Maharashtra, India, Maharashtra Data > Nuance. Full time

    About Data>Nuance Data>Nuance is a global privacy, data protection, and AI governance consultancy, trusted by 1,000+ organizations worldwide.We operate across Barcelona, Bangalore, and Dubai (expanding soon), with a dedicated team of data protection, governance, and regulatory specialists.We specialize in:Outsourced DPO ServicesData Protection Consultancy &...

  • Data Engineer

    3 hours ago

    Pune, Maharashtra, India, Maharashtra HCLTech Full time

    We’re Expanding Our Digital & Data Talent Pool Are you a professional ready to take on enterprise-scale transformation journey? We are on a mass hiring drive and looking for experts who can bring innovation and depth to our projects.Role we are looking for: GCP Data Engineering Minimum Experience level that is needed - We are looking for minimum >6 years...

  • Data Engineer

    3 hours ago

    Pune, Maharashtra, India, Maharashtra EXL Full time

    We are seeking an experienced GCP Data Engineer with strong expertise in designing, building, and managing scalable data pipelines and analytical solutions on Google Cloud Platform (GCP). The ideal candidate will have a solid background in big data technologies, data warehousing, and cloud-based data architecture, with hands-on experience using GCP-native...

  • Data Engineer

    3 hours ago

    Pune, Maharashtra, India, Maharashtra KPI Partners Full time

    KPI Partners is seeking a talented Data Engineer with expertise in STIBO (STEP) development to join our dynamic team. The ideal candidate will be responsible for designing, developing, and implementing data solutions that enhance our data management practices and streamline processes.Key Responsibilities:Develop, implement, and maintain STIBO STEP data...

  • Data Engineer

    3 hours ago

    Pune, Maharashtra, India, Maharashtra Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS !!!TCS is hiring for AWS Data engineer, please find the below JD..Experience range – 6 to 8 yearsLocation - Hyderabad/ Pune/ IndoreSkills Required - AWS, Pyspark, EMR, GlueRole& Responsibilities – · AMI Rehydration, Cloud formation, EMR, Glue, S3, Lambda/Python, ELB and other AWS components knowledge · AWS monitoring...

  • Lead Data Engineer

    3 hours ago

    Pune, Maharashtra, India, Maharashtra Searce Inc Full time

    Who we are?Searce means ‘a fine sieve’ & indicates ‘to refine, to analyze, to improve’. It signifies our way of working: To improve to the finest degree of excellence, ‘solving for better’ every time. Searcians are passionate improvers & solvers who love to question the status quo.The primary purpose of all of us, at Searce, is driving...

  • Data Engineer

    3 hours ago

    Pune, Maharashtra, India, Maharashtra Pixeldust Technologies Full time

    Role Overview:We are seeking a highly skilled Data Engineer - GCP with 6–8 years of experience in designing, developing, and managing enterprise data solutions on Google Cloud Platform (GCP). The ideal candidate will have a strong background in cloud data architecture, data warehousing, big data processing, and data integration, with proven expertise in...

  • Lead Data Engineer

    3 hours ago

    Pune, Maharashtra, India, Maharashtra EXL Full time

    About the RoleWe are seeking a highly skilled Lead Data Engineer to drive end-to-end data engineering initiatives, lead cross-functional teams, and deliver scalable, cloud-based data solutions. The ideal candidate will bring deep technical expertise, strong leadership, and the ability to collaborate effectively with global stakeholders in a fast-paced...

  • Data Engineer

    3 hours ago

    Pune, Maharashtra, India, Maharashtra Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS !!!TCS is hiring for Azure Data Engineer, please find the below JD..Experience range – 6 to 8 yearsLocation - Pune, Kolkata, Mumbai, BangaloreSkills Required - Azure Databricks, Azure data factory, Azure synapse, PysparkRequired Competencies:1) Strong design and data solutioning skills2) Pyspark hands-on experience with...

  • Data and Analytics Engineering Lead

    3 hours ago

    Pune, Maharashtra, India, Maharashtra Orbia Pune Full time

    We are currently looking for Data and Analytics Engineering Lead to join a fast growing product based Organisation in Pune.If you are someone who can develop a comprehensive technical plan for the migration, including data ingestion, transformation, storage, and access control in Azure's Data Factory and data lake! We'd love to hear from you.Role: Data and...