Big Data Developer

3 weeks ago

Gurgaon, Haryana, India Talentonlease Pvt ltd Full time

Key Responsibilities :

Data System Development & Optimization :

- Design, develop, and implement efficient data pipelines and data storage solutions using Big Data technologies.

- Build robust, scalable, and high-performance data processing systems using distributed frameworks such as Apache Spark (Core, Streaming, SQL), Flink, or Storm.

- Optimize and troubleshoot data processing jobs to ensure high throughput, low latency, and resource efficiency.

- Work on ClickHouse or similar OLAP (Online Analytical Processing) systems for high-performance analytics on large datasets.

Cloud Data Engineering :

- Develop and manage data solutions using AWS Cloud services such as S3, Redshift, Glue, Kinesis, EMR, and other cloud-based data services.

- Integrate cloud data solutions with on-premises data infrastructure, ensuring seamless data movement and access.

- Implement and optimize cloud-based ETL (Extract, Transform, Load) processes in AWS.

- Ensure data security, integrity, and compliance with industry standards in cloud environments.

Programming & Development :

- Utilize Java, Scala, and Python for developing data processing applications, algorithms, and custom solutions.

- Build reusable code and libraries for future use, ensuring modularity and scalability of data solutions.

- Write complex queries and data transformation logic to meet business and analytical needs.

Collaboration & Solution Design :

- Collaborate with cross-functional teams, including Data Scientists, Data Analysts, and Business Analysts, to understand requirements and design data architectures.

- Work with the product and engineering teams to define and implement data solutions that are aligned with business goals.

- Provide expertise in the integration of data systems with other business applications.

Performance & Scalability :

- Monitor and improve the performance of data systems and queries, ensuring the scalability of solutions to handle growing data volumes.

- Conduct performance tuning and troubleshooting of large-scale distributed data processing jobs to optimize speed and cost-effectiveness.

- Leverage cloud-native tools and frameworks to ensure efficient data storage, processing, and access.

Documentation & Reporting :

- Create and maintain detailed technical documentation for data systems, including architecture, processes, and workflows.

- Report on the status of data processing projects, providing insights into data trends, system performance, and areas for improvement.

Technical Qualifications :

Core Skills :

Big Data Technologies :

- Proficient in Apache Spark (Core, Streaming, SQL), Flink, Storm, or other distributed data processing frameworks.

- Strong knowledge of ClickHouse or other OLAP systems for large-scale data analytics and querying.

- Experience in designing and building high-performance data pipelines.

Cloud Data Services (AWS) :

- Expertise in AWS services: S3, Redshift, EMR, Glue, Kinesis, Lambda, and CloudFormation.

- Experience with cloud-based ETL processes and data storage/management in AWS.

Programming Languages :

- Expertise in Java, Scala, and Python for developing data-driven applications and algorithms.

- Proficient in building distributed systems and data processing frameworks in these languages.

Data Storage & Management :

- Familiar with HDFS, NoSQL databases (e.g, Cassandra, HBase), and data warehouses like Redshift and BigQuery.

- Knowledge of data lake architectures and cloud storage solutions.

Data Integration & Transformation :

- Skilled in ETL processes and frameworks for ingesting, transforming, and storing large volumes of data.

- Proficient in SQL and NoSQL query languages for data retrieval and processing.

Performance & Optimization :

- Strong understanding of performance tuning and optimization techniques for large-scale distributed systems.

- Experience with data partitioning, sharding, and optimizing big data storage and access.

Version Control & Automation :

- Familiar with Git and CI/CD pipelines for version control and automated deployment.

Preferred Qualifications :

- Experience with additional Big Data technologies such as Kafka, Hive, Presto, or Druid.

- Familiarity with containerization tools like Docker and orchestration platforms like Kubernetes.

- Experience working with Data Lakes and Machine Learning models in production environments.

- Familiar with DevOps practices and cloud-native development tools.

Education & Work Experience :

- Bachelor's/Master's degree in Computer Science, Engineering, or a related technical field.

- 8+ years of IT experience, with at least 5+ years in data-related technologies.

- Minimum 3+ years of experience in AWS Cloud services for data engineering.

- Proven experience in working with distributed systems and Big Data technologies like Apache Spark, ClickHouse, or similar.

Soft Skills :

- Strong problem-solving skills and the ability to work under pressure.

- Ability to work independently and as part of a collaborative team.

- Strong communication skills, with the ability to explain complex technical concepts to non-technical stakeholders.

- Highly organized, detail-oriented, and results-driven

(ref:hirist.tech)

  • Big Data Developer

    7 days ago

    Gurgaon, Haryana, India Impetus Full time

    Location: Indore, Pune, Noida, Gurugram and Bangalore Experience: 2- 5 years Description: We are looking for Big Data Developer having minimum experience of 2 year in big data ecosystem, passionate about technology, motivated for continuous learning and an individual who views every client interaction as an opportunity to create an exceptional customer...

  • Big Data Developer

    1 week ago

    Gurgaon, Haryana, India Impetus Full time

    Location: Indore, Pune, Noida, Gurugram and BangaloreExperience: 2- 5 yearsDescription:We are looking for Big Data Developer having minimum experience of 2 year in big data ecosystem, passionate about technology, motivated for continuous learning and an individual who views every client interaction as an opportunity to create an exceptional customer...

  • Big Data Developer

    2 weeks ago

    Gurgaon, Haryana, India Coforge Full time

    Please share resume at nishika.arora@coforge.com with below details: Experience- CTC- Expected CTC- Notice Period- Current Location- Preferred Location- Job Details SKILLS - BIG Data Engineer Mandatory Skills- Big Data, Hadoop, Hive, Spark , Scala , Airflow (good to have) Experience - 5+ years Job Location - Gurugram Detailed Job Description- Bachelor's or...

  • Senior Big Data Developer

    7 days ago

    Gurgaon, Haryana, India EXL Service Full time

    Job DescriptionThe Senior Big Data Developer will be responsible for building scalable data pipelines using Python, Spark, and other big data technologies. You will work closely with cross-functional teams to ensure smooth project execution and deliver high-quality results.

  • Big Data Solutions Developer

    1 day ago

    Gurgaon, Haryana, India Talentonlease Pvt ltd Full time

    **Company Overview:**Talentonlease Pvt ltd is a leading company in the industry, providing innovative solutions to our clients.**Job Description:**We are seeking a highly skilled Big Data Solutions Developer to join our team. As a Big Data Solutions Developer, you will be responsible for developing and managing data solutions using AWS Cloud...

  • Java Big Data Developer

    2 weeks ago

    Gurgaon, Haryana, India Virtusa Full time

    Java Big Data Developer - CREQ188171 Description Responsibilities include, but are not limited to - Develops and tests software, including ongoing refactoring of code, and drives continuous improvement in code structure and quality Primary focus is spent writing code, API specs, conducting code reviews and testing in ongoing sprints, or doing proof of...

  • Big Data Developer

    3 weeks ago

    Gurgaon, Haryana, India Intraedge Technologies Ltd. Full time

    Job Description :- A Bachelor's degree in computer science, computer engineering, other technical discipline, or equivalent work experience- 6 - 9 years of software development experience. Hands-on expertise with application design, software development, and automated testing.- Strong programming knowledge in SQL, Pyspark, Data processing- Hands-on...

  • Big Data Engineer

    6 days ago

    Gurgaon, Haryana, India Impetus Full time

    Company Overview:Impetus is a leading technology consulting firm that helps businesses leverage the power of big data to drive innovation and growth.We are seeking a highly skilled Big Data Developer to join our team, who has a strong background in big data technologies and excellent analytical skills.

  • Big Data Developer Lead

    5 days ago

    Gurgaon, Haryana, India Evalueserve Pvt. Ltd Full time

    Evalueserve Pvt. Ltd is a leading provider of business process management and IT services. We are looking for a talented Big Data Developer Lead to join our team.The Big Data Developer Lead will be responsible for designing, developing, and maintaining scalable data pipelines using Databricks and Apache Spark. This individual will work closely with our data...

  • Senior Big Data Developer

    7 days ago

    Gurgaon, Haryana, India Synechron Full time

    Company OverviewSynechron is a global consulting firm that offers innovative digital solutions to its clients. With a strong emphasis on Artificial Intelligence, Consulting, Digital, Cloud & Dev Ops, Data, and Software Engineering, we serve a wide range of industries including financial services and technology firms. Our company culture values diversity,...

  • Big Data Solutions Developer

    7 days ago

    Gurgaon, Haryana, India SheThink Pvt. Ltd. Full time

    Senior Data Engineer RoleSheThink Pvt. Ltd. is seeking a highly experienced Sr. Data Engineer to lead the development of our data engineering team. In this role, you will be responsible for designing, developing, and maintaining complex data pipelines and architectures using big data technologies like Spark, Azure, and Databricks. Your expertise in ETL/ELT...

  • Big Data Analyst

    1 week ago

    Gurgaon, Haryana, India EXL Full time

    Are you passionate about harnessing the power of data to drive business growth? We're looking for a seasoned Data Insights Consultant to join our team at EXL. In this role, you'll collaborate with cross-functional teams to support data-driven decision-making.Using your expertise in SQL and Big Data technologies (Hive, Hadoop, Spark), you'll develop Python...

  • Big Data Solutions Expert

    1 week ago

    Gurgaon, Haryana, India Sirius AI Full time

    Job Requirements:We are seeking a Big Data Solutions Expert to join our team at Sirius AI. As a key member of our data engineering team, you will be responsible for designing and implementing scalable big data solutions on AWS to support our clients' business needs.Key Responsibilities:Design and develop efficient big data solutions on AWS using services...

  • Big Data Engineer

    5 days ago

    Gurgaon, Haryana, India Coders Brain Technology Private Limited Full time

    Company OverviewCoders Brain Technology Private Limited is a leading technology firm dedicated to harnessing the power of data. Job Description We are seeking an experienced Technical Lead with expertise in Big Data engineering initiatives. As part of our team, you will lead and manage projects ensuring robust and scalable solutions. Your responsibilities...

  • Big Data Engineering Professional

    6 days ago

    Gurgaon, Haryana, India BayOne Solutions Full time

    About UsBayOne Solutions is a software consulting organization based in the US with over 12 years of experience. We specialize in providing talent and solutions to help companies grow their business.Job SummaryWe are seeking a highly skilled Big Data Engineering Professional to join our team. As a Big Data Engineering Professional, you will be responsible...

  • Big Data Solutions Architect

    6 days ago

    Gurgaon, Haryana, India BayOne Solutions Full time

    Our Expectations:We are looking for a highly skilled and experienced individual to join our team as a Big Data Solutions Architect. The ideal candidate will have a strong background in data engineering, data warehousing, and big data analytics.Responsibilities:Design and develop modern data engineering/data warehousing/data lakes technologies on cloud...

  • Big Data Analytics Specialist

    7 days ago

    Gurgaon, Haryana, India Synechron Full time

    About the RoleWe are seeking a highly skilled professional to join our team as a Senior Big Data Developer. As a key member of our global consulting firm, you will have the opportunity to work on cutting-edge digital solutions and contribute to the success of our clients.Job DescriptionThis role involves working on big data projects using various...

  • Big Data Engineer Manager

    1 week ago

    Gurgaon, Haryana, India PAR Technology Full time

    Required Skills and QualificationsTo be successful in this role, you will need 7+ years of experience as a Big Data engineering professional, developing scalable big data solutions. You must have an advanced degree in computer science, engineering, or a related field. Additionally, you will need to have extensive knowledge of cloud technologies, preferably...

  • Senior Big Data Engineer

    2 days ago

    Gurgaon, Haryana, India Pylon Management Consulting Full time

    About the Role :We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. In this role, you will be responsible for designing, developing, and maintaining our data infrastructure and pipelines. You will work with large datasets, implement scalable solutions, and ensure data quality and reliability. The ideal candidate...

  • Big Data Solutions Expert

    3 days ago

    Gurgaon, Haryana, India Synechron Technologies Private Limited Full time

    We are seeking a highly skilled Big Data Solutions Expert to join our team at Synechron Technologies Private Limited. As a key player in our organization, you will be responsible for designing and implementing data pipelines, data lakes, and visualization solutions to create business value.We're looking for someone with 6-8 years of experience in data and...