Hive/NiFi Engineer
3 days ago
Responsibilities :
- Design and develop data pipelines using Apache Nifi for ingesting, transforming, and loading data into our Big Data platform.
- Utilize Apache Hive for efficient data warehousing and querying large datasets.
- Leverage Apache Spark for large-scale data processing and analytics.
- Collaborate directly with clients to understand their business needs and translate those needs into effective Big Data solutions.
- Participate in discussions around data models and architecture for the Spectrum Mobile Data Hub platform.
- Fine-tune Spark-submit scripts for optimal performance and efficiency.
- Continuously optimize Hive SQL queries to ensure fast and reliable data retrieval.
- Create and manage Hive and Kudu tables for efficient data storage and retrieval.
- Utilize XML data sources for effective data loading into Big Data platforms.
Requirements :
- Bachelor's degree in Computer Science, Information Technology, or a related field (preferred).
- 3-4 years of experience working with Big Data technologies like Hive, Nifi, and Spark.
- Strong understanding of data warehousing concepts and data modeling principles.
- Proficiency in SQL and experience with HiveQL for querying large datasets.
- Experience with data ingestion and transformation tools like Apache Nifi.
- Experience with distributed processing frameworks like Apache Spark is a plus.
- Excellent communication and collaboration skills with the ability to interact effectively with clients.
- Strong analytical and problem-solving skills
-
Data Engineer
2 months ago
Anywhere in India/Multiple Locations/Bangalore, IN Mobile Programming LLC Full timeJob Description :Position : Data EngineerExperience : 6+ yearsWork Mode : Work from OfficeLocation : BangaloreExperience in Nifi or Tibco is mandatory.Mandatory Skills : ETL, DevOps platform, Nifi or TibcoPlease note : This position is focused on development rather than migration. We are seeking an experienced Data Engineer to join our team. As a Data...
-
Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations/Bangalore, IN Mobile Programming LLC Full timeJob Description :Position : Data EngineerExperience : 6+ yearsWork Mode : Work from OfficeLocation : BangaloreExperience in Nifi or Tibco is mandatory.Mandatory Skills : ETL, DevOps platform, Nifi or TibcoPlease note : This position is focused on development rather than migration. We are seeking an experienced Data Engineer to join our team. As a Data...
-
GCP Data Engineer
2 months ago
Pune/Hyderabad/Anywhere in India/Multiple Locations, IN Huquo Full timeJob Description :Must-Have :- 5+ Years of Experience in Data Engineering and building and maintaining large-scale data pipelines.- Experience with designing and implementing a large-scale Data-Lake on Cloud Infrastructure - Strong technical expertise in Python and SQL- Extremely well-versed in Google Compute Platform including BigQuery, Cloud Storage, Cloud...
-
GCP Data Engineer
4 weeks ago
Pune/Hyderabad/Anywhere in India/Multiple Locations, IN Huquo Full timeJob Description :Must-Have :- 5+ Years of Experience in Data Engineering and building and maintaining large-scale data pipelines.- Experience with designing and implementing a large-scale Data-Lake on Cloud Infrastructure - Strong technical expertise in Python and SQL- Extremely well-versed in Google Compute Platform including BigQuery, Cloud Storage, Cloud...
-
Big Data Architect
2 months ago
Metros/Anywhere in India/Multiple Locations, IN Xcel Hire Management Solutions Full timeJob description : - The decision-making power for data analysis and he/she should also possess the quality of architecting the massive data- Should have skills in big data tools and technologies; it includes technologies like the Hadoop, accumulo, MapReduce, Hive, HBase, panoply and redshift- Overall 10 years of experience as Principal Consultant, Subject...
-
Big Data Architect
4 weeks ago
Metros/Anywhere in India/Multiple Locations, IN Xcel Hire Management Solutions Full timeJob description : - The decision-making power for data analysis and he/she should also possess the quality of architecting the massive data- Should have skills in big data tools and technologies; it includes technologies like the Hadoop, accumulo, MapReduce, Hive, HBase, panoply and redshift- Overall 10 years of experience as Principal Consultant, Subject...
-
Senior Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations, IN BLossomJob Full timeSenior Data Engineers (Scala, Spark/PySpark, SQL) x2 headcounts- Primarily experts of Spark 3.X, SCALA, Delta lake implementation, streaming solution implementationfor IOT in Spark streaming. Kafka expertise.Any experience with MFG BI, DWH and Datalake implementation would be a bonus. Location : WFH from IndiaDomain : Semicon- 1x new headcount in 7-10 yrs...
-
Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations, IN Shankheshwar Enterprises Full timeOur client is a leading player in the Digital Transformation space have 17 centers in USA and IndiaJob Description :- Analyze and organize raw data, developing and maintaining data systems and pipelines, and collaborating closely with data scientists and business analysts to ensure data quality and reliability.- Quickly grasp and translate business...
-
Azure Data Engineer
1 month ago
Anywhere in India/Multiple Locations, IN TekNinjas Full timeRole : Data Engineer - Azure Synapse Analytics - Experience in Data engineering projects using Microsoft Azure platform (Min 2-3 projects)- Strong expertise in data engineering tools and storage such as Azure ADLS Gen2, Blob storage- Experience implementing automated Synapse pipelines- Ability to implement Synapse pipelines for data integration ETL/ELT using...
-
Azure Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations, IN TekNinjas Full timeRole : Data Engineer - Azure Synapse Analytics - Experience in Data engineering projects using Microsoft Azure platform (Min 2-3 projects)- Strong expertise in data engineering tools and storage such as Azure ADLS Gen2, Blob storage- Experience implementing automated Synapse pipelines- Ability to implement Synapse pipelines for data integration ETL/ELT using...
-
Data Engineer
3 days ago
Anywhere in India/Multiple Locations, IN Optivalue tek consulting Full timeJob Description :As an agile data engineer & software developer, need to work together with business users and members of the development team to provide quality systems that improve productivity, decrease operational risk, enable the business to expand and make users smile. These are high profile applications that offer you the opportunity to craft the...
-
Big Data/Databricks Engineer
4 weeks ago
Metros/Anywhere in India/Multiple Locations, IN WEN Full timeRole : Big Data + DatabricksResponsibilities :- Design, develop, and implement scalable Big Data solutions using the Hadoop ecosystem.- Work closely with data scientists and analysts to understand their requirements and translate them into technical solutions.- Develop and optimize Spark and Hive scripts to process, transform, and analyze large datasets.-...
-
Senior Data Engineer
1 month ago
Anywhere in India/Multiple Locations, IN Spruce IT Pvt. Ltd. Full timePan India. Experience : 5-8 years. JD : Azure Databricks : Role : Data Engineer (Sr).- The Senior Data Engineer will be responsible for the build of Enterprise Data platform.- Setting up the data pipelines that are scalable, robust and resilient and build pipelines to validate, ingest, normalize/enrich and business-specific processing of healthcare data.-...
-
Senior Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations, IN Spruce IT Pvt. Ltd. Full timePan India. Experience : 5-8 years. JD : Azure Databricks : Role : Data Engineer (Sr).- The Senior Data Engineer will be responsible for the build of Enterprise Data platform.- Setting up the data pipelines that are scalable, robust and resilient and build pipelines to validate, ingest, normalize/enrich and business-specific processing of healthcare data.-...
-
Lead Big Data Engineer
2 months ago
Anywhere in India/Multiple Locations, IN Grownex HR Solution Full timeRole : Lead Big Data EngineerWork Mode : Remote(5 Days working)The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate...
-
Lead Big Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations, IN Grownex HR Solution Full timeRole : Lead Big Data EngineerWork Mode : Remote(5 Days working)The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate...
-
Big Data Engineer
4 weeks ago
Anywhere in India/Multiple Locations, IN D-TechWorks Pvt Ltd Full timeWe are hiring for Big Data AI/ML Engineer - Remote , India Position : Big Data AI/ML EngineerExperience : 6 to 9 YrsLocation : Remote We are seeking a talented Data Engineer with AI & ML knowledge to join our team. As Data Engineer or ML Ops Engineer, your primary responsibility will be to develop & integrate ML solutions that focus on technology...
-
Platform Engineer
2 months ago
Chennai/Anywhere in India/Multiple Locations, IN Mobile Programming LLC Full timeJob Description :Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development plans,...
-
Platform Engineer
4 weeks ago
Chennai/Anywhere in India/Multiple Locations, IN Mobile Programming LLC Full timeJob Description :Job Description : Platform EngineerTitle : Platform Engineer Location : Chennai Work Mode : Hybrid (Remote and Chennai Office) Experience : 6+ years Responsibilities :- Administer systems including configuration, monitoring and maintenance of Application servers;- Able to discern stakeholder needs, effectively communicate development plans,...
-
Manager/Senior Manager
2 months ago
Anywhere in India/Multiple Locations, IN HuQuo Consulting Pvt. Ltd. Full timeAs Manager, Data Engineering, you will be responsible for translating client requirements into design, architecting, and implementing GCP Cloud-based big data solutions for clients. Your role will be focused on delivering high-quality solutions by independently driving design discussions related to Data Ingestion, Transformation & Consumption, Data Storage...