Big Data Engineer

4 weeks ago


GoaMumbaiJammu KashmirJammuSrinagarPondicherryJaipurLucknowVaranasiBanarasPatnaRanchi, IN ATech Full time

Designation: BIG DATA ENGINEER

Job Description:

Your Role and Responsibilities:

- Understand a data warehousing solution and able to work independently in such an environment

- Responsible in Project development and delivery experience of a few good size projects

- Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements.

- Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing data driven organization.

- Coordinate data access and security to enable data scientists and analysts to easily access to data whenever they need too.

- Experiencing developing scalable Big Data applications or solutions on distributed platforms

- Able to partner with others in solving complex problems by taking a broad perspective to identify innovative solutions

- Strong skills building positive relationships across Product and Engineering.

- Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders

- Able to quickly pick up new programming languages, technologies, and frameworks

- Experience working in Agile and Scrum development process

- Experience working in a fast-paced, results-oriented environment

- Experience in Amazon Web Services (AWS) or other cloud platform tools

- Experience working with Data warehousing tools, including Dynamo DB, SQL, Amazon Redshift, and Snowflake

- Experience architecting data product in Streaming, Serverless and Microservices Architecture and platform.

- Experience working with Data platforms, including EMR, Data Bricks etc

- Experience working with distributed technology tools, including Spark, Presto, Scala, Python, Databricks, Airflow

- Developed the Pysprk code for AWS Glue jobs and for EMR.. Worked on scalable distributed data system using Hadoop ecosystem in AWS EMR, MapR distribution..

- Developed Python and pyspark programs for data analysis.. Good working experience with python to develop Custom Framework for generating of rules (just like rules engine).

- Developed Hadoop streaming Jobs using python for integrating python API supported applications..

- Developed Python code to gather the data from HBase and designs the solution to implement using Pyspark. Apache Spark DataFrames/RDD's were used to apply business transformations and utilized Hive Context objects to perform read/write operations..

- Re - write some Hive queries to Spark SQL to reduce the overall batch time

Required Technical and Professional Expertise:

- First and most important- Sound understanding of data structures & SQL concepts and experience in writing complex SQL especially around OLAP systems

- Sound knowledge of the ETL tool like informatica, 5+ years of experience, Big Data technologies'' like Hadoop ecosystem, its various components, along with different tools including Spark, Hive, Sqoop,etc..

- In-depth knowledge of MPP/distributed systems

Preferred Technical and Professional Expertise:

- The ability to write precise, scalable, and high-performance code

- The ability to write precise, scalable, and high-performance code

- Knowledge/Exposure in data modeling with OLAP (Optional)

(ref:hirist.tech)
  • Valiance Solutions

    4 weeks ago


    Noida/Bangalore/Mumbai, IN Valiance Analytics Pvt. Ltd. Full time

    About Us : Valiance is a global AI & Data analytics firm helping clients build cutting-edge technology solutions for digital transformation. We work with some of the marquee brands across India, US and APAC to build transformative solutions for Credit Risk, Fraud, Predictive Maintenance, Quality Inspection, Data lake, IOT analytics etc. Our team comprises...

  • Big Data Engineer

    3 weeks ago


    Rajasthan/Karnataka/Metros/Jaipur/Udaipur/Bangalore/Raipur, IN ATech Full time

    Hiring For Big Data Engineer (Spark, Scala, AWS)Job description :- Spark, SCALA, AWS OR Spark, python with AWS lambda exposure.- Design, build, optimize and support new and existing data models and ETL processes based on our clients business requirements.- Build, deploy and manage data infrastructure that can adequately handle the needs of a rapidly growing...

  • IT Manager

    4 weeks ago


    Gujarat/Ahmedabad/Rajkot/Surat/Vadodara/Baroda/Guwahati/Srinagar/Jammu & Kashmir/Jammu/Hyderabad/Tel, IN Aricent Full time

    Role : IT Data Engineer - ManagerJob description :- The Data Engineer Manager role will be an integral part of an Agile team focused on delivering best in class business intelligence- Under the general supervision of senior management and the Data Integration Chapter Lead in the Enterprise Data Product Portfolio, you will be working with key customers to...

  • Big Data Developer

    4 weeks ago


    Bangalore/Gurgaon/Gurugram/Pune/Chennai/Coimbatore/Mumbai/Cochin/Kochi/Noida, IN SP Software Pvt. Ltd. Full time

    Job Description :- Should have hands on experience in Software development- Responsible for designing, deploying, and managing high quality data solutions in the AWS cloud ecosystem.- Create and maintain optimal data architecture pipeline.- Collaborate with various onsite and client stakeholders in Agile environment to identify data engineering requirements...


  • Andhra Pradesh/Vijayawada/Vishakhapatnam/Vizag/Jammu & Kashmir/Jammu/Srinagar, IN ATS India Full time

    Role : Lead Full Stack DeveloperIndustry Type : IT Services Department : Engineering Employment Type : Full Time, PermanentJob description :- Work with distributed development teams and product managers to ideate software solutions.- Developing front end website architecture.- Designing user interactions on web pages.- Developing back-end website...

  • Data Engineer

    1 week ago


    Bangalore/Pune/Kolkata/Gurgaon/Gurugram/Mumbai/Ahmedabad, IN Sterlite Technologies Limited Full time

    Role as a Data engineer :- Develop and build AI/ML model ready data ponds using Azure Databricks Delta lake- Improve upon the data ingestion models, ETL jobs, and alarm to maintain data integrity and data availability.- Partner with AI/ML product teams proactively identify the data needs for rapid experimentation, stable production deployment and ensure the...

  • Manual Test Engineer

    4 weeks ago


    Goa/Kolkata/Andhra Pradesh/Vishakhapatnam/Vizag/Vijayawada/Guntur/Jammu & Kashmir/Srinagar, IN ATech Full time

    Manual Test EngineerJob Description :- Responsible for the overall quality of IT projects and changes through execution of all aspects of the Software Development and Testing Life Cycle- Conduct a feasibility study, and assist with automation utilizing current frameworks and tools.- Streamline the testing process by introducing automation, leveraging tools,...

  • Data Engineer

    4 weeks ago


    Chennai/Mumbai/Pune, IN iMind Your Business Solutions Private Limited Full time

    Company Description :iMind Business Consulting Inc. is a strategy and talent consulting company headquartered in the US with offices in the UK, US and India. Role Description :This is a full-time hybrid role for one of our niche clients headquartered in San Francisco, US, with offices in India. Leaders in analytics and cloud applications, headquartered in...


  • Hyderabad/Bangalore/Chennai/Jaipur/Mumbai, IN Talentxo Full time

    Job Description :- Work with the product team in developing new features, identifying solution gaps etc.- Interest and aptitude in learning new technologies - Big Data, no SQL databases, Elastic Search, Mongo DB, devops.- Create detailed designs, solutions and validate with internal engineering and customer teams, establish a good network of relationships...

  • ETL Developer

    4 weeks ago


    Jaipur/Udaipur/Rajasthan/Karnataka/Bangalore/Mysore, IN ATech Full time

    Job : ETL DeveloperSkills Required :- Experience in the database, data integration, and data migration- Good experience (3+ years) in development using databases such as Oracle, MS SQL Server, Teradata, etc.- Minimum of 2+ years of hands-on experience with Amazon Redshift.- Strong knowledge of the AWS environment and service knowledge with S3 storage...

  • Data Engineer

    4 weeks ago


    Bangalore/Mumbai/Pune/Chennai/Kolkata/Noida, IN Change leaders Full time

    Primary Skill : - GCP + Python + SQLOR- GCP + Java + SQLNP : Immediate to 35 Days maxJob Location : Mumbai / Pune / Chennai / Bangalore / Hyderabad / Kolkata / Noida- Bachelor's degree in computer science or a related discipline- 6+ years of applicable engineering experience- Experience in GCP, Big Query- Strong proficiency in Python with an emphasis in...

  • Senior Data Scientist

    4 weeks ago


    Gurgaon/Gurugram/Mumbai, IN Invokhr Full time

    Position: Senior Data ScientistDesired Experience: 8+ yearsYour team :You will act as a key member of the Data consulting team, working directly with the partners and senior stakeholders of the clients. You will collaboratively work with interdisciplinary scientists, IT and engineering professionals across the organization to solve critical problems and...

  • Senior Data Engineer

    4 weeks ago


    Anywhere in India/Multiple Locations/Mumbai, IN Grorapid labs Full time

    Job Bachelor's or advanced degree in Computer Science, Information Technology, or related field.- Proven experience as a Back-End Data Engineer or similar role.- Proficiency in database design and management (e.g., SQL, NoSQL).- Strong programming skills in languages such as Python, Java, or Scala.- Experience with ETL tools and data integration...


  • Bihar/Jharkhand/Maharashtra/Pondicherry/Coimbatore/Patna/Aurangabad/Ranchi/Mumbai/Navi Mumbai/Pune/N, IN ATech Full time

    Job Profile : Spark ( Pyspark ) DeveloperIndustry Type : IT Services Job description :- The developer must have sound knowledge in Apache Spark and Python programming.- Deep experience in developing data processing tasks using pySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations.-...

  • Data Engineering Lead

    2 weeks ago


    Lucknow/Bangalore, IN Carnation Infotech Private Limited Full time

    Job Description :We are looking for a strong Leader with 10+ years of overall IT experience and having at least 5 years of experience in the data engineering domain. In this highly visible role, you will serve as the technical expert responsible for the design and implementation of the data architecture and stack that ensures timely delivery of analytics to...

  • Lead Data Architect

    4 weeks ago


    Gurgaon/Gurugram/Mumbai, IN Huquo Full time

    Job Description :While working closely with users and providing a high level of customer service in a friendly, professional manner, the successful candidate must be able to :- Design multi-year IT data-driven transformation/delivery roadmaps qualifying strategic business opportunities - Define long-term strategic goals for data architecture vision,...


  • Bangalore/Chennai/Hyderabad/Mumbai/Pune/Noida, IN ACZ Global Private Limited Full time

    Azure Data ArchitectMandatory Skills : Solution Architecture - Pyspark + Databricks + Adf + Synapse is mandatoryJob Description :We are seeking a highly skilled and experienced Azure Data Architect to join our team. As an Azure Data Architect, you will play a key role in designing and implementing data solutions on the Microsoft Azure platform. The ideal...

  • Python Data Engineer

    4 weeks ago


    Bangalore/Hyderabad/Pune/Mumbai/Chennai, IN Swift Strategic Staff Solutions INC Full time

    Job Description :We are seeking a talented and experienced Python Data Engineer to join our data team and play a crucial role in building, maintaining, and optimizing data pipelines for our cloud-based data warehouse.Responsibilities :- Design, develop, and implement data pipelines for ingesting, transforming, and loading data into Snowflake using Python...

  • Data Engineer

    4 weeks ago


    Pune/Mumbai/Hyderabad/Chennai/Bangalore, IN RapidBraiins Full time

    Position Overview :As a Talend - Data Engineering Specialist, you will play a key role in designing, developing, and maintaining data pipelines using Talend, AWS, and SQL. You will work closely with cross-functional teams to gather requirements, design data solutions, and implement scalable and efficient ETL processes. Additionally, you will leverage your...


  • Kolkata/Jharkhand/Goa/Jamshedpur/Ranchi/Patna/Pune/Bangalore, IN Aricent Full time

    Profile : Desktop Support EngineerJob Description :- Handling all IT products like, Server, Lease line, Network, Firewall, PRI, etc.- Providing support and maintenance through the ticketing tool- IT Asset handling in entire and tracking with record for maintenance for Asset software- Knowledge of SonicWall Firewall- Configuring & troubleshooting Firewall,...