Big Data Engineer

7 days ago


Delhi, Delhi, India Programming Full time

Job Title :
Big Data Engineer

Location :
Remote

About Us :
We are a leading specializing in software development . Our team is dedicated to leveraging cutting-edge technologies to deliver scalable and efficient solutions to our clients globally.

Role Overview :


We are seeking a highly skilled Big Data Engineer with a strong background in designing and developing robust solutions using big data technologies.

The ideal candidate will be proficient in Python, PySpark, and have hands-on experience with cloud platforms such as Azure, AWS, or GCP.


This role requires expertise in data manipulation, querying across various database types, and a strong focus on building scalable ingestion pipelines.


Key Responsibilities :

  • Design, develop, and deploy scalable solutions using big data technologies.
  • Utilize Python and PySpark for efficient data processing and analysis.
  • Work with cloud platforms (Azure, AWS, GCP) to deploy and optimize data solutions.
  • Query and manipulate data from various databases including relational and big data platforms.
  • Write effective and maintainable unit and integration tests for ingestion pipelines.
  • Implement static analysis, code quality tools, and CI/CD pipelines to automate software delivery.
  • Collaborate effectively with crossfunctional teams to deliver highquality solutions in a fastpaced, dynamic environment.
Provide technical leadership, mentorship, and guidance to junior team members.

Requirements :

  • Bachelor's degree in Computer Science, Engineering, or related field (Master's preferred).
  • 6+ years of experience in design and development with a focus on big data technologies.
  • Proficiency in Python and PySpark for data processing and analysis.
  • 2+ years of handson development experience with cloud technologies (Azure, AWS, GCP).
  • Strong experience in querying and manipulating data from various DB (relational and big data).
  • Demonstrated expertise in writing unit and integration tests for ingestion pipelines.
  • Familiarity with static analysis tools, code quality tools, and building CI/CD pipelines.
  • Excellent communication skills and ability to work collaboratively in a team environment.
  • Proven ability to thrive in hightraffic and largescale software product environments.

Preferred Qualifications :

  • Experience with streaming data technologies (e.g., Kafka, Apache Spark Streaming).
  • Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes).
  • Certification in relevant technologies (e.g., AWS Certified Big Data Specialty, Azure Data Engineer Associate).

Benefits :

  • Competitive salary and performancebased incentives.
  • Comprehensive health benefits package.
  • Flexible work hours and remote work options.
  • Professional development opportunities and certifications.
  • Vibrant company culture with teambuilding activities and social events.
)
  • Big Data Engineer

    1 week ago


    Delhi, Delhi, India Awign Expert Full time

    Job DescriptionThis is a remote positionJob Title:Big Data Solutions EngineerReporting to: Director, Product ManagementLocation: India (Remote)Our Ideal CandidateOur company is seeking a talented Big Data Solutions Engineer to join our team.The ideal candidate will be responsible for designing, developing, and implementing our proprietary Bigdata-powered Fin...

  • Big Data Engineer

    7 days ago


    Delhi, Delhi, India OmegaHires Full time

    Job Title: Big Data Engineer (Cornerstone to Lumi migration)Duration: Full-time on OmegaHires Payroll Location: NCR (Hybrid - 3 days Onsite per week) Should understand the Credit Card domain and good to have knowledge on Compliance & Operational Risk . Good analytics experience to do data discovery, analyze the gaps in data, partner with Data Owners, build...


  • Delhi, Delhi, India Databricks Full time

    As aSolutions Architect(Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be atechnical advisorinternally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve tangible...


  • Delhi, Delhi, India Leading MNC Full time

    Looking for aFreelance Big Data Solutions Engineerto join a team of rockstar developers. The candidate should have aminimum of 4+experience.There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you You will report into...


  • Delhi, Delhi, India Databricks Full time

    As a Solutions Architect (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a technical advisor internally to the sales team, and work with the product team as an advocate of your customers in the field. You will help our customers to achieve tangible...


  • Delhi, Delhi, India Rackspace Full time

    About the Role:We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. The ideal candidate will have a strong background in developing and scaling both stream and batch processing systems, and a solid understanding of public cloud technologies, especially GCP. This role involves working in a remote environment,...


  • Delhi, Delhi, India Data Engineer Academy Full time

    Job Title: Operations AnalystLocation : RemoteCompany Overview:Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary to...


  • Delhi, Delhi, India Data Engineer Academy Full time

    Job Title: Operations AnalystLocation : RemoteCompany Overview:Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary to...

  • ETL Big Data

    7 days ago


    Delhi, Delhi, India LTIMindtree Full time

    Hi AllLooking for ETL Big data testerExperience- 3-8Location-ChennaiWorking experience in Big Data Testing or ETL TestingVery Good in SQL or HiveGood in LinuxWell versed in Agile processShould work independently and interact with customerBig Data Application Knowledge and Tele com background would be an added advantage


  • Delhi, Delhi, India XPT Software Australia Pty Ltd Full time

    Job DescriptionJob Description:1.Understanding the non-functional requirements2.Able to develop Test Plans, TCRindependently.3.Design and Development of Test scripts4.Well Versed in Microfocus Load Runner-not limited to Web/WS but also in other Protocols-Java, Citrix, MQ, Oracle etc.5.JMeter Experience desired.6.Experience at least in one of Monitoring tools...

  • Big Data Developer

    7 days ago


    Delhi, Delhi, India Impetus Full time

    Experience Required: 4-5 yrsLocation:GurgaonImmediate Joiners onlyJOB DESCRIPTION:Skillset:Should have strong experience in big data technologies.Basic skills in data pipeline – experience in building data pipelines for data migrationBig Data – Pyspark, SparkStrong command on SQL & HiveProgramming Language: Any - Java/Python/Scala


  • Delhi, Delhi, India Tiger Analytics Full time

    Job TitleAzure Big Data ArchitectLocation : Chennai, Hyderabad, BangaloreWho we areTiger Analytics is a global leader in AI and analytics, helping Fortune 1000 companies solve their toughest challenges.We offer full-stack AI and analytics services & solutions to empower businesses to achieve real outcomes and value at scale.We are on a mission to push the...


  • Delhi, Delhi, India Tiger Analytics Full time

    Job TitleAzure Big Data ArchitectLocation: Chennai, Hyderabad, BangaloreWho we areTiger Analytics is a global leader in AI and analytics, helping Fortune 1000 companies solve their toughest challenges. We offer full-stack AI and analytics services & solutions to empower businesses to achieve real outcomes and value at scale. We are on a mission to push the...

  • Data Engineer –

    7 days ago


    Delhi, Delhi, India HARMAN International Full time

    HARMAN's engineers and designers are creative, purposeful and agile. As part of this team, you'll combine your technical expertise with innovative ideas to help drive cutting-edge solutions in the car, enterprise and connected ecosystem. Every day, you will push the boundaries of creative design, and HARMAN is committed to providing you with the...

  • Data Engineer

    7 days ago


    Delhi, Delhi, India Kenan Management Consultants Pvt Ltd Full time

    Job DescriptionThe role requires data ingestion, processing and extraction of data to & from Hadoop Cluster onlyusing Informatica BDM. It requires extensive experience in ETL, SQL, Scoop, Spark, Hive, Hadoop &KAFKA. Knowledge of PySpark will be preferableJob Title Data EngineerExperienceMinimum 4-5 years of Experience in Development inInformatica Big Data...

  • Data Engineer

    1 week ago


    Delhi, Delhi, India Tech Mahindra Full time

    Over all exp : 3-5yrsNotice : ImmediateMust Have Skills:3-5 yrs, of Strong expertise inPythonfor data engineering tasks (ETL/ELT).3-5yrs, of Experience with big data tools is desirable:Spark, Spark-Streaming, Kafka, Debeziumetc.experience with two or more of this big data streaming technology is mandatory:Apache Spark Streaming, Apache Flink, Apache Kafka,...

  • Big Data Developer

    1 week ago


    Delhi, Delhi, India Teamware Solutions a division of Quantum Leap Consulting Pvt. Ltd Full time

    Hello Connections,Greetings From Teamware Solutions...Excellent Opportunity for Our MNC ClientExperience : 4+ Years(Mandatory)Location :ChennaiInterview Mode:F2FInterview Date:1-Jun-2024Note :we are looking for Immediate Joiners (References Acceptable)Role/Skill Set :Big data Developer,Spark/Scala#immediatejoinersInterested Candidates can share their updated...

  • Big Data Engineer

    7 days ago


    Delhi, Delhi, India Sureminds Solutions Pvt. Ltd. Full time

    Job Description have skills : 1. Handson experience of Spark, Scala and conversant with SQLGood knowledge on Hadoop (Oozie)Reverse engineer the SQL queries, Scala code to understand functionalityCapable of identifying, analysing and interpret patterns and trends in complex data setsShould have strong experience on AWS (EMR, S3)Has worked on creating...

  • Data Engineer

    7 days ago


    Delhi, Delhi, India Tech Mahindra Full time

    Over all exp : 3-5yrsNotice : ImmediateMust Have Skills:3-5 yrs, of Strong expertise inPythonfor data engineering tasks (ETL/ELT).3-5yrs, of Experience with big data tools is desirable:Spark, Spark-Streaming, Kafka, Debeziumetc.experience withtwo or moreof this big data streaming technology is mandatory:Apache Spark Streaming, Apache Flink, Apache Kafka,...

  • Big Data Developer

    7 days ago


    Delhi, Delhi, India Teamware Solutions a division of Quantum Leap Consulting Pvt. Ltd Full time

    Hello Connections,Greetings From Teamware Solutions...Excellent Opportunity for Our MNC ClientExperience : 4+ Years[Mandatory]Location :ChennaiInterview Mode:F2FInterview Date:1-Jun-2024Note :we are looking for Immediate Joiners [References Acceptable]Role/Skill Set :Big data Developer,Spark/Scala#immediatejoinersInterested Candidates can share their updated...