Big Data Engineer

1 week ago

India Careerfit Full time

Job Title: Big Data Solutions Engineer

About the Company: Our company is a leading tech firm specializing in innovative data solutions that empower businesses to make informed decisions.

Responsibilities :

Solution Design :

  • Design, develop, and maintain scalable big data solutions and data pipelines for efficient data ingestion, processing, storage, and analysis.
  • Collaborate with data scientists, analysts, and business stakeholders to identify data requirements, use cases, and business objectives.
  • Translate business needs into technical designs and select appropriate big data technologies for optimal performance.

Stakeholder Collaboration :

  • Partner with data scientists, analysts, and business stakeholders to understand data needs, answer technical questions, and provide data access solutions.
  • Communicate complex technical concepts clearly and concisely to nontechnical audiences.
  • Foster strong working relationships across departments to ensure successful data-driven projects.

Data Processing :

  • Implement and optimize data ingestion pipelines using tools like Apache Kafka to acquire data from various sources in real-time or batch mode.
  • Utilize big data technologies such as Apache Hadoop, Spark, and HBase to efficiently process and store large datasets.
  • Develop and maintain data quality checks and data cleansing routines to ensure data integrity and consistency.

Data Transformation :

  • Design and maintain data models and schemas for structured and unstructured data.
  • Develop and implement ETL (Extract, Transform, Load) processes to transform raw data into usable formats for analysis.
  • Perform data cleansing, normalization, and deduplication tasks to improve data quality for downstream applications.

Performance Optimization :

  • Monitor the performance, scalability, and reliability of big data solutions.
  • Identify and troubleshoot performance bottlenecks to ensure efficient data processing and access.
  • Continuously optimize data pipelines and infrastructure to meet evolving data volumes and processing demands.

Data Governance :

  • Champion data governance best practices throughout the data lifecycle.
  • Implement robust data security measures to safeguard sensitive information.
  • Ensure adherence to data privacy regulations and compliance requirements.

Continuous Learning :

  • Stay up-to-date with emerging big data technologies, trends, and best practices through ongoing learning initiatives.
  • Participate in professional development activities, such as conferences and workshops, to enhance your technical skillset.

Requirements :

  • Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
  • Proven experience in big data engineering or related roles (minimum X years).
  • Proficiency in programming languages like Java, Python, or Scala.
  • Solid understanding of distributed computing frameworks (Hadoop, Spark).
  • Experience with data modeling, schema design, and ETL processes.
  • Excellent problem-solving, analytical, and communication skills.
  • Ability to work effectively in a team environment.

Skills :

  • Big Data Engineering : Proficient in designing, developing, and maintaining big data solutions and pipelines.
  • Data Processing : Skilled in implementing and optimizing data ingestion, processing, and storage using big data technologies.
  • Collaboration : Ability to collaborate effectively with cross-functional teams.
  • Problem-Solving : Strong problem-solving skills to identify and resolve technical challenges.

Preferred Qualifications :

  • Certification in big data technologies (Cloudera Certified Professional, Hortonworks Certified Developer).
  • Experience with cloud platforms (AWS, Azure, GCP) for big data deployments.
  • Experience with data visualization tools (Tableau, Power BI).
  • Experience with scripting languages (Bash, Shell).

  • Big Data Engineer

    3 weeks ago

    india Awign Expert Full time

    Job Description This is a remote position. Job Title: Big Data Solutions Engineer Reporting to: Director, Product Management Location: India (Remote) Our Ideal Candidate Our company is seeking a talented Big Data Solutions Engineer to join our team. The ideal candidate will be responsible for designing, developing, and...

  • Lead Big Data Engineer

    1 week ago

    India Grownex HR Solution Full time

    Role : Lead Big Data Engineer Work Mode : Remote(5 Days working) The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate...

  • Big Data Solutions Engineer

    1 week ago

    India Gethyr Full time

    Our company is seeking a talented Big Data Solutions Engineer to join our team. The ideal candidate will be responsible for designing, developing, and implementing our proprietary Bigdata-powered Fin Crime solution for our clients. The candidate will work closely with the software development team, data scientists, and product managers to understand...

  • Big Data Engineer

    2 weeks ago

    india Evnek Technologies Pvt Ltd Full time

    Job Description This is a remote position. Duration:  6 MonthLocation:  RemoteTimings: Full Time (As per company timings)Notice Period: within 15 days or immediate joinerExperience: 4-6 Years   Responsibilities Create and maintain data pipelines that transfer data from select locations (maintaining security protocols) into our Big Data financial...

  • Big Data Engineer

    1 week ago

    India ATech Full time

    Designation:BIG DATA ENGINEERJob Description:Your Role and Responsibilities: Understand a data warehousing solution and able to work independently in such an environment Responsible in Project development and delivery experience of a few good size projects Design, build, optimize and support new and existing data models and ETL processes based on our clients...

  • Big Data Engineer

    1 week ago

    Anywhere in India/Multiple Locations D-TechWorks Pvt Ltd Full time

    We are hiring for Big Data AI/ML Engineer - Remote , India Position : Big Data AI/ML EngineerExperience : 6 to 9 YrsLocation : Remote We are seeking a talented Data Engineer with AI & ML knowledge to join our team. As Data Engineer or ML Ops Engineer, your primary responsibility will be to develop & integrate ML solutions that focus on technology...

  • Data Engineer

    3 weeks ago

    india Evnek Technologies Pvt Ltd Full time

    Job Description This is a remote position. Role: Data Engineer  Experience: 2+Years Notice period: Immediate Location: Anywhere from India Job Type: Full Time Responsibilities: Designing & building data architectures including the design & set-up of scalable (big) data lakes (full cloud or hybrid cloud), scheduled transformations &...

  • Data Engineer

    1 month ago

    india Evnek Technologies Pvt Ltd Full time

    Job Description This is a remote position. Role: Data Engineer  Experience: 2+Years Notice period: Immediate Location: Anywhere from India Job Type: Full Time Responsibilities: Designing & building data architectures including the design & set-up of scalable (big) data lakes (full cloud or hybrid cloud), scheduled transformations &...

  • Big Data Architect

    1 week ago

    India ARR Recruitment Solutions Full time

    About the Role : We are seeking a highly experienced Big Data Architect to join our team and lead the design, development, and implementation of our big data solutions. This role is ideal for a candidate with a deep understanding of big data technologies and a proven track record of delivering scalable and robust big data architectures.Responsibilities :...

  • Big Data Engineer

    5 days ago

    india Programming.com Full time

    Job Title : Big Data EngineerLocation : RemoteAbout Us :We are a leading specializing in software development . Our team is dedicated to leveraging cutting-edge technologies to deliver scalable and efficient solutions to our clients globally.Role Overview :We are seeking a highly skilled Big Data Engineer with a strong background in designing and developing...

  • GCP Big data Engineer

    1 week ago

    India IKAI Technology Solutions Full time

    We are seeking a talented Big Data + GCP Engineer to join our dynamic team. In this role, you will be responsible for designing, implementing, and maintaining scalable data pipelines and infrastructure on Google Cloud Platform (GCP). You will collaborate closely with cross-functional teams to ensure that our data platform meets the needs of our business...

  • GCP Big data Engineer

    1 week ago

    india IKAI Technology Solutions Full time

    We are seeking a talented Big Data + GCP Engineer to join our dynamic team. In this role, you will be responsible for designing, implementing, and maintaining scalable data pipelines and infrastructure on Google Cloud Platform (GCP). You will collaborate closely with cross-functional teams to ensure that our data platform meets the needs of our business...

  • GCP Big data Engineer

    1 week ago

    India IKAI Technology Solutions Full time

    We are seeking a talented Big Data + GCP Engineer to join our dynamic team. In this role, you will be responsible for designing, implementing, and maintaining scalable data pipelines and infrastructure on Google Cloud Platform (GCP). You will collaborate closely with cross-functional teams to ensure that our data platform meets the needs of our business...

  • Big Data Architect

    1 week ago

    India Swift Strategic Staff Solutions INC Full time

    Responsibilities :Data Ingestion & Processing : Design and build robust data pipelines using tools like Apache Spark, Flink, or Kafka to ingest, transform, and process massive datasets from various sources.Big Data Technologies : Utilize your expertise in big data frameworks like Hadoop, YARN, and HDFS to manage, store, and analyze large volumes of data...

  • Big Data Quality Assurance Engineer

    1 week ago

    India Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...

  • Lead Big Data/Data Engineer

    1 day ago

    india TechTiv Software Solutions Full time

    The Opportunity : Digit88 development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate would be joining an existing team of outstanding data...

  • Freelance Big Data Engineer

    1 week ago

    India Leading MNC Full time

    Looking for a Freelance Big Data Solutions Engineer to join a team of rockstar developers. The candidate should have a minimum of 4+experience. There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you You will report...

  • Big Data Engineer

    7 days ago

    india Taiyō.AI Full time

    We empower the people who build the world. Taiyō.AI is the world's largest infrastructure construction data-mesh technology and the first AI platform for the global infrastructure construction industry. Our clients include some of the largest construction firms, suppliers, and government. About The Team We are looking for Big Data Engineer to help support...

  • Big Data Engineer

    6 days ago

    India Taiyō.AI Full time

    We empower the people who build the world.Taiyō.AI is the world's largest infrastructure construction data-mesh technology and the first AI platform for the global infrastructure construction industry. Our clients include some of the largest construction firms, suppliers, and government. About The TeamWe are looking for Big Data Engineer to help support and...

  • Data Engineer –

    3 weeks ago

    india HARMAN International Full time

    HARMAN’s engineers and designers are creative, purposeful and agile. As part of this team, you’ll combine your technical expertise with innovative ideas to help drive cutting-edge solutions in the car, enterprise and connected ecosystem. Every day, you will push the boundaries of creative design, and HARMAN is committed to providing you with the...