Big Data Specialist
2 months ago
Experience Required: 3+ years.
Location: PAN India
Responsibilities:
Design, develop and deploy solutions using different tools, design principles and conventions.
Assemble large, complex data sets that meet functional / non-functional business requirements.
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources and ‘big data’ technologies.
Optimize data ingestion and processing performance by tuning configuration parameters, optimizing SQL queries, and implementing parallel processing techniques.
Monitor pipeline performance and resource utilization to identify bottlenecks and optimize system resources allocation.
Set up monitoring and alerting systems to track pipeline health, detect anomalies, and notify stakeholders of potential issues or failures.
Configure logging and metrics collection to facilitate troubleshooting and performance analysis.
Understand existing processes and facilitate change requirements as part of a structured change control process.
Maintain proper documentation for the solutions, test procedures and scenarios during UAT and Production phase.
Understand and implement Pipeline Development, Data Quality and Governance, Monitoring and Alerting, Data Virtualization and Query Optimization, Data Source Integration, Data Catalog Management, Security and Access Control, Integration with Applications and Ecosystems, Data Ingestion and Management
Coordinate with process owners and business to understand the as-is process and design the automation process flow.
Produce high-quality code resulting from knowledge of the latest frameworks, code peer review, and automated unit test scripts.
Collaborate with other developers and team members.
-
Big Data QA Specialist
7 days ago
Mumbai, Maharashtra, India Brainhunter Recruiting Pvt Ltd Full timeJob Title: Big Data QA SpecialistWe are seeking a highly skilled Big Data QA Specialist to join our team at Brainhunter Recruiting Pvt Ltd. The ideal candidate should possess expertise in automation testing using Java or Python programming languages.Key Responsibilities:Automation Testing:Develop and maintain automated test scripts using Java or Python for...
-
Big data specialist
1 week ago
Mumbai, India Deloitte Full timeWe are looking for Candidates who are having experience in Big Data.Experience Required: 3+ years.Location: PAN IndiaResponsibilities:- Design, develop and deploy solutions using different tools, design principles and conventions.- Assemble large, complex data sets that meet functional / non-functional business requirements.- Identify, design, and implement...
-
Big Data Developer
6 days ago
Mumbai, Maharashtra, India Pegasus InfoCorp Full timeAbout the Role:Pegasus InfoCorp is seeking a highly skilled Big Data Developer to join our team.As a Big Data Developer, you will be responsible for building solutions for customers using Big Data technologies. You will work on designing, developing, and implementing Big Data solutions that meet our customers' needs. You will also be responsible for...
-
Big Data Engineer
1 week ago
Mumbai, Maharashtra, India Smartavya Analytica Private Limited Full timeKey Responsibilities:As a Data Processing Expert at Smartavya Analytica Private Limited, you will be responsible for developing, testing, and implementing Big Data projects using Hadoop, Spark, and Hive. Your expertise in setting up Hadoop services, implementing ETL/ELT pipelines, and working with Terabytes of data ingestion and processing from varied...
-
Big Data Developer
4 weeks ago
Mumbai, Maharashtra, India krtrimaIQ Cognitive Solutions Full timekrtrimaIQ Cognitive Solutions is seeking a highly skilled Big Data Developer to join our team. As a Big Data Developer, you will play a crucial role in designing, developing, and maintaining scalable data processing pipelines using Kafka, PySpark, Python/Scala, and Spark.The RoleDesign and develop efficient data processing pipelines using Kafka, PySpark, and...
-
Big Data Developer
4 days ago
Mumbai, Maharashtra, India Tata Consultancy Services Full timeJob Role: Big Data DeveloperWe are looking for a qualified Big Data Developer to join our team at Tata Consultancy Services.Responsibilities:Ingest data from diverse sources and develop ETL jobs using Spark, PySpark, Hive, and HBase.Perform impact analysis and provide estimates.Oversee end-to-end deliverables.Create project plans and work on implementation...
-
Big Data Developer
3 weeks ago
Mumbai, Maharashtra, India Synechron Technologies Pvt. Ltd. Full timeAbout the Role:We are seeking a skilled Big Data Developer to join our team at Synechron Technologies Pvt. Ltd. The ideal candidate will have proficiency in one or more general-purpose big data languages, including Hive and Spark.Key Responsibilities:Design and develop big data solutions using Hive, Java, or SparkWork with UNIX, SQL Server, and JavaDevelop...
-
Big Data QA
1 week ago
Mumbai, India Brainhunter Recruiting Pvt Ltd Full timeJob Title: Big Data QA Engineer We are seeking a highly skilled and motivated Big Data QA Engineer with expertise in automation testing using Java or Python programming languages. The ideal candidate should possess a strong background in data validation, data verification, PySpark, pipeline monitoring, software...
-
Big Data Engineer
3 weeks ago
Chennai/Hyderabad/Bangalore/Coimbatore/Mumbai/Delhi/Pune, India GAIUS HYPERLOCAL PRIVATE LIMITED Full timeJob Title: Big Data EngineerWe are seeking a highly skilled Big Data Engineer to join our team at GAIUS HYPERLOCAL PRIVATE LIMITED. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining scalable and efficient big data solutions.Key Responsibilities:Design and develop scalable big data architectures and pipelines.Develop...
-
Big Data Architect
6 days ago
Mumbai, Maharashtra, India Vertoz Full timeJob OverviewWe are seeking a skilled Data Engineer with Hadoop Experience to join our team at Vertoz. The ideal candidate will be responsible for designing, developing, and maintaining our data pipeline architecture, ensuring seamless data flow and collection for cross-functional teams.The selected candidate will have hands-on experience with Hadoop, Spark,...
-
Big Data Engineer
2 weeks ago
Mumbai, Maharashtra, India Synechron Technologies Pvt. Ltd. Full timeAbout SynechronSynechron is a global consulting firm that specializes in digital transformation and innovation. Our team of experts combines creativity and technology to deliver industry-leading digital solutions.Job RoleThe Hadoop Developer will work on designing and developing big data solutions using Hive, Java, and Spark. The ideal candidate should have...
-
Big Data Engineer
4 weeks ago
Mumbai, Maharashtra, India Smartavya Analytica Private Limited Full timeJob DescriptionSmartavya Analytica Private Limited is seeking a highly skilled Hadoop Developer to join our team.Key ResponsibilitiesDesign and implement large-scale Big Data projects using Hadoop, Spark, and Hive.Lead the development of ETL/ELT pipelines and data ingestion processes.Collaborate with cross-functional teams to deliver high-quality data...
-
Senior Big Data Engineer
3 weeks ago
Mumbai, Maharashtra, India AEXONIC TECHNOLOGIES PRIVATE LIMITED Full timeJob Title: Senior Big Data EngineerAbout the Role:We are seeking a highly skilled Senior Big Data Engineer to join our team at AEXONIC TECHNOLOGIES PRIVATE LIMITED. The successful candidate will be responsible for designing, building, and maintaining robust data pipelines that support large-scale data processing and analytics.Key Responsibilities:- Design,...
-
Big Data Engineer
4 weeks ago
Mumbai, Maharashtra, India Snaphunt Full timeThe OpportunityWe are seeking an experienced Big Data Developer to join our team at Snaphunt. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining scalable data processing pipelines using Kafka, PySpark, Python/Scala, and Spark.The RoleDesign and implement robust data processing solutions using...
-
Big Data Manager
2 weeks ago
Mumbai, Maharashtra, India Protegrity USA Full timeProtegrity USA is seeking a seasoned technology professional to fill the role of Manager, Engineering Big Data. The ideal candidate will have 10+ years of experience in developing enterprise software and leading/managing a team. This position is a remote role and requires adaptability to learn and enhance skills in the domain of data security alongside the...
-
Big Data Developer
3 months ago
Mumbai, India Snaphunt Full timeThe OfferWork within a company with a solid track record of successExcellent career development opportunitiesGreat work environmentThe JobDesign, develop, and maintain scalable data processing pipelines using Kafka, PySpark, Python/Scala, and Spark.Work extensively with the Kafka and Hadoop ecosystem, including HDFS, Hive, and other related...
-
Big Data Developer
3 months ago
Mumbai, India krtrimaIQ Cognitive Solutions Full timekrtrimaIQ having deep expertise in Technology domains Cognitive Process Automation, Data Engineering, Statistical Analysis and Optimization, Data Science, Natural Language Processing and Computer Vision, is well prepared to transform organizations to Intelligent Enterprise.The RoleDesign, develop, and maintain scalable data processing pipelines using Kafka,...
-
Big Data Solutions Architect
4 days ago
Mumbai, Maharashtra, India Pegasus InfoCorp Full timeAbout the RolePegasus InfoCorp seeks an experienced Big Data Solutions Architect to join our team and drive innovation in the field of Hadoop development.Design and implement scalable Big Data solutions using Hadoop, Spark, and other related technologies.Collaborate with cross-functional teams to understand business needs and develop effective...
-
Big Data Architect
2 weeks ago
Bangalore/Pune/Mumbai/Chennai, India a client of change leaders Full timeJob SummaryWe are seeking a highly skilled Big Data Architect to join our team at a client of change leaders. As a key member of our data engineering team, you will be responsible for designing and implementing scalable Big Data applications and solutions on distributed platforms.Key ResponsibilitiesDesign and implement data pipelines using Snowflake,...
-
Big Data
3 months ago
Mumbai, India Overture Rede Private Limited Full timeThis is a remote position.Design& Develop Transformations aspects using the ELT frameworkto modernize the Ingestion pipelines and build data transformationsat scale Work closely with other data teams &Data Science teams and participate in the development of Ingestionpipelines Provide technical expertise in the areas ofdesign and implementation of Ratings...