Senior Data Engineer

2 weeks ago


Vijayawada, India Integrated Wireless Solutions Full time

Exp: 8+ Years

Required Skills: Databricks (Expert) + Pyspark (Advanced) + MongoDB (Advanced) + Scala (Understands)


As a Data Engineer (Cloud Native), you will be responsible for designing, developing, and maintaining our cloud-based data infrastructure and systems. Your primary focus will be on leveraging Azure storage, Databricks, NoSQL databases like MongoDB, and other cloud native technologies to build scalable and efficient data pipelines. You will collaborate closely with cross-functional teams, analysts, and software engineers, to ensure optimal data flow and availability for analytics and reporting purposes.


Experience Range:

8+ years of experience in relevant skills and technologies.


Responsibilities:

• Design, develop, and maintain cloud native data infrastructure and systems using Azure storage, Databricks, NoSQL databases like MongoDB, and related technologies.

• Develop and implement data ingestion and transformation processes to ensure efficient and accurate data flow from various sources into the data platform.

• Develop event based pipeline that process data in real-time and apply business logics

• Collaborate with cross-functional teams to understand data requirements and develop appropriate data models and schemas.

• Optimize data pipelines for performance, scalability, and reliability, ensuring high availability and low latency.

• Implement and maintain data security and privacy measures, adhering to industry best practices and compliance standards.

• Monitor and troubleshoot data pipelines, identifying and resolving issues proactively to ensure uninterrupted data flow.

• Work closely with analysts to understand data usage patterns and optimize data structures and schemas for analytics and reporting purposes.

• Implement and maintain data orchestration using tools like Azure DevOps and Kubernetes, ensuring smooth deployment and management of data processes.

• Stay up to date with the latest trends and advancements in cloud native data engineering technologies, evaluating and recommending new tools and techniques as needed.


Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field. A master's degree is a plus.

Extensive experience working as a Data Engineer, preferably in a cloud native environment.

Strong proficiency in Azure storage, Databricks, NoSQL databases like MongoDB, and related technologies.

Proven experience in data ingestion and transformation methods, including ETL/ELT processes.

• Solid understanding of data modeling and schema design principles.

• Familiarity with cloud-based deployment and orchestration tools like Azure DevOps and Kubernetes.

• Strong analytical and problem-solving skills, with the ability to troubleshoot and resolve complex data engineering issues.

• Excellent communication and collaboration skills, with the ability to work effectively in cross-functional teams.

• Experience with Agile development methodologies is a plus.

  • Experience with Metadata Driven Framework is a plus.
  • Familiarity with Data Marketplace and Data Mesh architecture is a plus.

• Relevant certifications in Azure or related technologies are highly desirable.



  • Vijayawada, India Accordion India Full time

    Company OverviewAccordion works at the intersection of sponsors and management teams throughout every stage of the investment lifecycle, providing hands-on, execution-focused support to enhance the capabilities of the CFO's office. So, what does it mean to work at Accordion? It means joining 1,000+ analytics, finance & technology experts in a high-growth,...

  • Senior Data Engineer

    4 weeks ago


    Vijayawada, India Segmentide Technology Solutions Full time

    Job Description:We are seeking a Senior Data Engineer with a strong background in Oracle PL/SQL, AWS Cloud, Databricks, and Data Modelling. This role requires an individual who can work independently and is capable of managing complex data projects from start to finish.Key Responsibilities:Design, develop, and optimize complex SQL queries and PL/SQL code for...

  • Data Engineer

    4 weeks ago


    Vijayawada, India Globus Systems Full time

    Company: AA GLOBUSDIGITAL INDIA PRIVATE LIMITEDAA GLOBUSDIGITAL INDIA PRIVATE LIMITED, is a wholly owned subsidiary of Globus Systems Inc US,Globus Systems was founded by industry executives who have been part of the IT services industry for the past 20 years and have seen it evolve and mature. We understand the challenges faced by organizations as they...

  • Data Engineer

    2 weeks ago


    Vijayawada, India Wipro Full time

    Job Description:Location-PAN IndiaExpertise in Data Engineering. Hands-on experience in design and development of big data platform.Deep understanding of modern data processing technology stacks: Spark, HBase, Hive and other Hadoop ecosystem technologies. Development using Scala. PysparkDeep understanding of streaming data architectures and technologies for...

  • AWS Data Engineer

    3 days ago


    Vijayawada, India FLYONIT Full time

    Job Summary:We are looking for a highly skilled AWS Data Engineer with 5-6 years of experience to join our dynamic team. The ideal candidate will have hands-on experience with AWS services, data engineering best practices, and a strong understanding of building, deploying, and maintaining data pipelines in the cloud. You will be responsible for designing and...


  • Vijayawada, India Anblicks Full time

    Lead Data Engineer :Exp : 10+Positions : 8Location : VijayawdaBachelor's or Master's degree in Computer Science, Information Systems, or a related field10+ years of experience in data engineering and data architectureStrong knowledge of data technologies, such as Snowflake, Databricks, Apache Spark, Hadoop, Dbt, Fivetran, Azure Data FactoryExperience...

  • Senior Sales Engineer

    2 weeks ago


    vijayawada, India FURNISYS Full time

    Job Description: Senior Sales Engineer Position: Senior Sales Engineer Location: Navi Mumbai, India  Salary: Competitive with performance-based incentives  Experience: 3+ years in industrial/technical sales  Reports to: Sales/Marketing Manager  About Furnisys:  Furnisys provides modular, functional furniture systems designed for industries such as...


  • Vijayawada, India Markovate Full time

    We are looking for a highly skilled and experienced Senior Technical Architect with over 15 years of expertise in software development, architecture, and strategic technology leadership, including a deep focus on AI technologies for one of our clients. The ideal candidate will have a strong background in AI/ML frameworks, data engineering, and cloud-based AI...


  • Vijayawada, India Xconnect Data Centers Pvt Ltd Full time

    Using network monitoring tools to identify and resolve issues such as bandwidth congestion, packet loss, latency, and downtime. They also optimize network performance to ensure smooth data transmission and minimal disruptions.Experience0 - 1 YearsNo. of Openings5EducationDiploma, Professional Degree, B.Tech, Any Bachelor DegreeRoleNetwork Engineer...


  • Vijayawada, India Design Cafe Full time

    Design Cafe is hiring Senior Design Engineer Address: No 25, 2nd floor, 15th Cross Rd, next to Ramakrishna Hospital, above Decathlon, Jaya Nagar 1st Block, Jayanagar 3rd Block East, Jayanagar, Bengaluru, Karnataka 560011Experience : 2 to 3 years of experience.Job Description:Creation of SKU/Article in Imos.Integrate new hardware components into the IMOS,...


  • Vijayawada, India Pi DATACENTERS Full time

    Job Title: Data Center Network Engineer L3Location: [Vijayawada]Job Type: Full-TimeDepartment: IT/Network OperationsReports to: Manager Cloud Operations.Job Summary:The Data Center Network Engineer L3 is responsible for the design, implementation, management, and optimization of data center network infrastructure. This role involves working with cutting-edge...


  • Vijayawada, India Pi DATACENTERS Full time

    Job Title: Data Center Network Engineer L3Location: [Vijayawada]Job Type: Full-TimeDepartment: IT/Network OperationsReports to: Manager Cloud Operations.Job Summary:The Data Center Network Engineer L3 is responsible for the design, implementation, management, and optimization of data center network infrastructure. This role involves working with cutting-edge...


  • Vijayawada, India Pi DATACENTERS Full time

    Job Title: Data Center Network Engineer L3 Location: (Vijayawada) Job Type: Full-Time Department: IT/Network Operations Reports to: Manager Cloud Operations. Job Summary: The Data Center Network Engineer L3 is responsible for the design, implementation, management, and optimization of data center network infrastructure. This role involves working with...


  • Vijayawada, India Wipro Full time

    Job Title: ATE Data Analysis EngineerExperience: 5yrs-10 yrsLocation: Bengaluru, Karnataka, IndiaDescription:Experience with Data analysis tools (JMP, Datapower, O+, PDF Solutions, etc.) for Vmin, Yield, Test Time & Bin Pareto AnalysisAbility to write scripts to automate and speed up data analysis in a repeated manner with different input data sets (mainly...


  • Vijayawada, India KONE Full time

    Description Customer relationship management • Proactively develops and creates customer relationships in his/her area of responsibility • In early phases of projects, drives solution selling methodology with architects and developers, consultants or other influencers to ensure KONE solutions are meeting customer’s needs and creating value...


  • vijayawada, India 282 KONE Elevator India P.Ltd Full time

    Description Customer relationship management• Proactively develops and creates customer relationships in his/her area of responsibility• In early phases of projects, drives solution selling methodology with architects and developers, consultants or other influencers to ensure KONE solutions are meeting customer’s needs and creating value...


  • Vijayawada, India People Tech Group Inc Full time

    HelloconnectionsWe are Hiring '' Lead /Data Architect Aws Devops Engineer '' and below are the requirements. This will be through ''People Tech Group Company''.Position: Aws Lead Devops Engineer/Data ArchitectCompany : People Tech Group ( Mnc)Mode of Work: WFOExperience: 6+yrsLocation : Bangalore OnlyMandatory Skill: Aws, cloud engineer, Devops ,terraform,...

  • Software Engineer

    4 weeks ago


    Vijayawada, India DDN Storage Full time

    DDN Storage is seeking great candidates to join our dynamic team of passionate customer-enabling technologists! This is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DDN Storage is a global market leader renowned for powering many of the world's most...


  • Vijayawada, India Cambium Networks Full time

    Scope of Responsibilities/ExpectationsAn ATE Software Engineer will be a vital member of an exceptional, highly motivated team of wireless system engineers. Our mission is to deliver carrier-grade wireless equipment to customers across various verticals, ranging from Wireless Internet Service Providers (WISPs) to Federal Governments. We seek candidates who...


  • vijayawada, India CIDROY TECHNOLOGIES LLP Full time

    Senior Software Architect (Node.js | AWS | Cloud) Location: Remote Experience: Minimum 5 years Job Description: Cidroy Technologies is seeking a highly skilled and experienced Senior Software Architect to join our dynamic team. This role is ideal for a passionate backend developer with a proven track record of designing and leading complex architectural...