![MindBrain](https://media.trabajo.org/img/noimg.jpg)
Senior Big Data Engineer
3 days ago
Job Title - Big Data Engineer.
Location - Remote.
Contractual - Duration 1 yr.
Responsibilities :
- Design, build, and maintain large-scale data processing systems (batch and streaming) using technologies such as Hadoop, Spark, Kafka, etc.
- Develop and optimize data pipelines, data ingestion processes, and data integration workflows.
- Implement data governance and security best practices to ensure data quality and integrity.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver actionable insights.
- Perform data profiling, analysis, and troubleshooting to ensure optimal performance and reliability of data systems.
- Monitor and maintain the health, performance, and reliability of data pipelines and infrastructure components.
- Implement best practices for data storage, data lake architecture, and data warehousing solutions.
- Work closely with DevOps and IT teams to deploy and manage big data solutions in cloud environments (AWS, Azure, GCP) or on-premises.
- Stay updated with emerging technologies and trends in big data and data engineering to drive continuous improvement.
Requirements :
- Proven experience as a Big Data Engineer or in a similar role designing and implementing large-scale data systems.
- Strong proficiency in programming languages such as Python, Java, Scala, etc.
- Hands-on experience with big data technologies such as Hadoop ecosystem (HDFS, Hive, HBase), Spark, Kafka, etc.
- Experience with data pipeline orchestration tools like Airflow, Luigi, or similar.
- Proficiency in SQL and NoSQL databases (e.g, MySQL, PostgreSQL, MongoDB).
- Familiarity with cloud platforms and services (AWS, Azure, GCP) and containerization technologies (Docker, Kubernetes).
- Solid understanding of data modeling, ETL processes, and data warehousing concepts.
- Excellent problem-solving skills and ability to work independently as well as in a team environment.
- Bachelor's degree in Computer Science, Engineering, or a related field; Master's degree preferred.
Exp 6 to 8 years. Location- Remote
-
Lead Big Data/Data Engineer
5 days ago
Remote/Bangalore/Pune, IN TechTiv Software Solutions Full timeThe Opportunity :Digit88 development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate would be joining an existing team of outstanding data...
-
Big Data/ETL Test Engineer
5 days ago
Remote, IN Gama Advisory Services LLP Full timeOverview :We are seeking a proficient Big Data/ETL Test Engineer with 4-6 years of experience in testing Big Data platforms and ETL processes, specifically using Hadoop and Hive technologies. As a Test Engineer, you will play a critical role in ensuring the quality and reliability of our Big Data solutions by designing and executing comprehensive test...
-
Java Developer
5 days ago
Remote, IN Gama Advisory Services LLP Full timeOverview :We are looking for a talented Java Developer with 2-3 years of experience in Big Data technologies to join our team. As a Java Developer specializing in Big Data, you will be responsible for designing, developing, and maintaining scalable applications and data pipelines using Java and Big Data frameworks. Your expertise in Java programming and Big...
-
Senior Associate Big Data Engineer
2 weeks ago
Remote, IN Suitable.AI Full timeRole Objective :Data Engineer will be responsible for expanding and optimizing our data and database architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building. The Data Engineer will support our software...
-
Senior Big Data Engineer
14 hours ago
Remote, IN MindBrain Full timeRoles & Responsibilities :- Sound knowledge in Spark architecture and distributed computing- Proficient in Spark - including RDD and Dataframes core functions, troubleshooting and performance tuning.- Good understanding in object-oriented concepts and hands on experience on Scala or on java with excellent programming logic and technique- Good in functional...
-
Data Scientist
5 days ago
Remote, IN Georgia IT, Inc. Full timeJob Description :- We are seeking a skilled Data Scientist / Data Analyst with a solid background in Python programming, SQL, and data visualization. - The ideal candidate will have at least 3 years of relevant experience in data science or data analysis, with specific expertise in NLP techniques and LLM. - This role requires a creative problem-solver who...
-
Aptus Data Labs
5 days ago
Remote, IN Aptus Data Labs Full timeDescription :We are seeking a highly skilled and experienced individual to join our team as a Data Governance expert with AWS knowledge. In this role, you will play a critical role in ensuring data integrity, quality, and security across our organization's data assets. As a data governance expert, your main responsibility will be to develop and implement...
-
Data Engineer
2 weeks ago
Remote, IN Suitable.AI Full timeRole Objective :Data Engineer will be responsible for expanding and optimizing our data and database architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building. The Data Engineer will support our software...
-
Senior Data Engineer
4 weeks ago
Remote, IN Nthinsight Full timeJob Description :We are seeking a skilled and experienced Data Engineer with expertise in Python, data warehousing, ETL (Extract, Transform, Load), and SQL. As a Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure to support the organization's data needs. You will collaborate with cross-functional...
-
Neurones IT
3 days ago
Remote, IN IT AISA Full timeJob Description :- As a Data engineer, you will be a part of a highly qualified team, leading the delivery of modern data platforms using Azure Services and Databricks.- Designing and delivering software using an agile and iterative approach based on Scrum or Kanban.- Following and contributing to improvement of client's software engineering practices...
-
Remote, IN Avalara Technologies Pvt ltd Full timeWe are seeking a top-performing, versatile, and passionate Senior Software Development Engineer in Test (SDET) with over 10 years of extensive experience in software testing for an open position in our Content Engineering department, where we curate all the tax content needs for various Avalara product suites. The ideal candidate will have solid expertise in...
-
Diatoz - Senior Data Engineer - ETL
3 days ago
Remote, IN Diatoz Solutions Full timeA Senior Data Engineer Specializing In Snowflake And ETL (Extract, Transform, Load) Processes Is Responsible For Designing, Implementing, And Maintaining Data Infrastructure And Pipelines To Ensure Data Is Efficiently Extracted From Various Sources, Transformed Into Usable Formats, And Loaded Into Data Warehouses Or Data Lakes.Responsibilities :- Developing...
-
Remote, IN Avalara Technologies Pvt ltd Full time- ONLY ENGINEERING GRADUATES ( COMPUTER SCIENCE AND RELATED FIELDS O NLY )- Candidates should be from product firms only ( No services firms )- Candidates from top tier institutes ( IIT / REC NIT ) & BITSWe are seeking a top-performing, versatile, and passionate Senior Software Development Engineer in Test (SDET) with over 10 years of extensive experience in...
-
Senior Data Engineer
14 hours ago
Remote, IN Hirextra Full timeResponsibilities :- Design, develop, and implement cloud-based data pipelines for efficient data movement and transformation.- Migrate data workloads from on-premise to cloud storage solutions (e.g, AWS S3).- Utilize cloud-based SQL and NoSQL databases (e.g, AWS Redshift, DynamoDB) for data storage and retrieval.- Develop and maintain data pipelines using...
-
Data Scientist/Analyst
4 weeks ago
Remote, IN Prime Infosoft Full timeWhat Does a Data Scientist Do?Data scientist roles and responsibilities include: - Data mining or extracting usable data from valuable data sources - Using machine learning tools to select features, create and optimize classifiers - Carrying out preprocessing of structured and unstructured data - Enhancing data collection procedures to include all relevant...
-
Senior Machine Learning Engineer
2 weeks ago
Remote, IN RiskInsight Consulting Pvt Ltd Full timeAbout the Role : We are seeking a highly skilled and experienced Senior Machine Learning Engineer to join our innovative team and play a pivotal role in developing and deploying impactful machine learning solutions. In this role, you will leverage your expertise in machine learning algorithms, data analysis, and software engineering to tackle complex...
-
Remote, IN Avalara Technologies Pvt ltd Full timeWe are seeking a highly skilled and experienced Senior AI & Machine Learning Technical Manager to lead our dynamic team in developing cutting-edge AI & ML solutions. This role is perfect for someone passionate about leveraging Artificial Intelligence and Machine Learning to solve complex problems, drive innovation, and create impactful products. You will be...
-
Senior Software Engineer
2 weeks ago
Remote, IN Blue Silicon Infotech Private Limited Full timeSenior Software Engineer PythonA Senior Software Engineer Python is a professional with extensive experience and expertise in Python programming language. They are responsible for designing, developing, and maintaining high-quality software applications using Python and related frameworks.Role and Responsibilities :- R&D on new technologies to solve problems...
-
Edelman Financial Engines
2 weeks ago
Remote, IN FENL IT SERVICES FINANCIAL ENGINES LLP Full timeSenior Staff Engineer - Salesforce Development At Edelman Financial Engines (EFE), we believe everyone deserves to move their financial life forward! We know that assets have the power to fund goals. Those numbers represent individual lifetimes filled with hard work and dreams for our clients and generations to follow. Our clients trust us to guide them...
-
Snowflake Developer
2 weeks ago
Remote, IN Flexiventures Pvt. Ltd Full timeWe are seeking a skilled Data Engineer with expertise in Snowflake to join our dynamic team. The ideal candidate will have a strong background in data engineering and a deep understanding of Snowflake architecture and development. You will be responsible for designing, building, and optimizing data pipelines and warehouse solutions within the Snowflake...