Data Engineer

4 days ago

Delhi, India Solytics Partners Full time

Company Profile:Solytics Partners is a Global Analytics firm, recognized with multiple industry awards for innovation and excellence. Our team comprises experts with deep knowledge in risk, analytics, AI/ML, AML/FCC, and fraud. By converging this expertise with cutting edge technologies like AI, Machine Learning, Generative AI, and Large Language Models (LLMs), we deliver powerful automated platforms and incisive point solutions. Our offerings enable clients to streamline and future-proof their risk, AML, and analytics processes, comply seamlessly with global regulations, and safeguard financial systems. Whether it’s solving complex challenges or driving operational efficiency, Solytics Partners is committed to empowering organizations with transformative tools to stay ahead in an evolving regulatory landscape.Job Title: Data Engineer (Web Scraping)Experience: 5 – 10 years of relevant experienceLocation & Timings: Pune – Work from office & Timing - 11:00 AM – 8:00 PMEducation Qualification: Masters or bachelor's in computer science or IT or in other relevant discipline from a reputed institute.Role Type: Permanent / Full TimeJob Description: We are seeking an experienced Data Engineering & Automation Lead to design, automate, and optimize large-scale data processing and web scraping pipelines. The role involves leading a team to build and maintain high-performance ETL workflows using Apache Airflow, Apache Spark, and AWS services, while integrating AI/NLP solutions powered by OpenAI GPT and other GenAI models for intelligent data extraction and analytics.Responsibilities:Design, automate, and maintain ETL and data processing pipelines using Apache Airflow and Apache Spark.Build, monitor, and optimize web scraping and data extraction workflows for global compliance and risk data sources.Lead and manage web scraping and data engineering teams, ensuring delivery excellence, code quality, and scalability.Create, design, and document automation workflows and secure data-sharing systems using AWS (Lambda, S3, API Gateway, SQS).Implement AI and NLP integrations using OpenAI GPT and GenAI models for intelligent data extraction, tagging, and analytical automation.Analyze large-scale datasets to identify quality gaps, improve accuracy, and optimize indexing and retrieval performance.Collaborate with Backend, DevOps, and Frontend teams for data modeling, monitoring, and visualization.Work closely with clients to gather and translate business requirements into scalable automation and analytics solutions.Author HLD/LLD documentation, mentor junior engineers, and continuously improve automation processes and data workflows.Required Skills:Programming: Python, SQL, JavaScriptData Engineering & Automation: Apache Airflow, Apache Spark, Web Scraping (Scrapy, Selenium), Pandas, NumPyDatabases & Storage: Elasticsearch, MongoDB, MySQLCloud & Backend: AWS (Lambda, S3, EC2, CloudWatch, SQS, SNS, EKS), Docker, Django, FlaskAI/ML & NLP : OpenAI GPT APIs, NER, Sentiment Analysis, Embeddings, Information ExtractionMonitoring & Tools: Grafana, Git, Postman, Jupyter, VS Code Good to HaveStrong understanding of Large Language Models (LLMs) and Generative AI for building intelligent data extraction and analytics agents.Familiarity with risk and compliance domains, including Sanctions, PEP (Politically Exposed Persons), and AMS (Adverse Media Screening) data and processes.Soft Skills:Leadership & Team MentoringProblem-Solving & Analytical ThinkingClear Technical CommunicationCross-functional Collaboration

  • Senior Data Engineer

    1 week ago

    Delhi, India Eucloid Data Solutions Full time

    Job Description:Eucloid is looking for a Senior Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and building of...

  • Data Analytics Lead

    23 hours ago

    delhi, India NTT DATA Full time

    Build, manage, and foster a high-functioning team of data engineers and Data analysts. Collaborate with business and technical teams to capture and prioritize platform ingestion requirements. Experience of working with manufacturing industry in building a centralized data platform for self service reporting. Lead the data analytics team members, providing...

  • Senior Data Engineer

    3 days ago

    New Delhi, India Eucloid Data Solutions Full time

    Job Description:Eucloid is looking for a Senior Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to designing and building of...

  • Senior Data Engineer

    2 days ago

    Delhi, NCR, India MchooversPacific Data Integrators Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Title: Senior Data Engineer (IDMC-CDI)Primary Skills: IDMC (CDI), ETL ConceptsMode of Work: Remote/WFHExperience: 8-15 yearsShift Timing: Open to work in EST shift (5PM to 2AM IST)About UsWe are PDI (Pacific Data Integrators), a leading provider of Data Management, Analytics, and Cloud solutions. As an Informatica Platinum Partner and a reseller of various...

  • Azure Data Engineer

    19 hours ago

    Delhi, India Mars Data Insights Full time

    HiringAzure Data Engineer(Azure / MS Fabric / SAP BW/BFC ) positioninremotelocationsSkills:Microsoft Fabric (MS Fabric), SAP BW/BFC Data Integration, Apache Kafka, Azure cloud services, Azure Functions, Docker Containers, and Kafka for real-time data processing, python, DW, Data Gov & Security, data modelling, ETL/ELT processes, and performance tuning, CI/CD...

  • Azure Data Engineer

    2 days ago

    Delhi, India Mars Data Insights Full time

    Hiring Azure Data Engineer (Azure / MS Fabric / SAP BW/BFC ) position in remote locationsSkills: Microsoft Fabric (MS Fabric), SAP BW/BFC Data Integration, Apache Kafka, Azure cloud services, Azure Functions, Docker Containers, and Kafka for real-time data processing, python, DW, Data Gov & Security, data modelling, ETL/ELT processes, and...

  • Senior Data Scientist

    3 weeks ago

    New Delhi, India Crayon Data Full time

    Role: Sr Data ScientistExperience level: 5 to 7 yearsLocation: ChennaiWhy Crayon? Why now?Crayon is transforming into an AI first company, and every Crayon (that’s what we call ourselves!) is undergoing a journey of upskilling and expanding their capabilities in the AI space.We're building an organization where AI is not a department—it’s a way of...

  • Senior Data Scientist

    3 weeks ago

    New Delhi, India Crayon Data Full time

    Role:Sr Data Scientist Experience level:5 to 7 years Location:ChennaiWhy Crayon? Why now? Crayon is transforming into an AI first company, and every Crayon (that’s what we call ourselves!) is undergoing a journey of upskilling and expanding their capabilities in the AI space.We're building an organization where AI is not a department—it’s a way of...

  • Technical Specialist

    13 hours ago

    New Delhi, India NTT DATA Full time

    Job Description Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive. Your day at NTT...

  • Data Analytics Lead

    2 weeks ago

    Delhi, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Analytics Lead to join our team in Delhi, Delhi (IN-DL), India (IN). Experience: 8+ Years Location: Delhi NCR...