Data Engineering Specialist

4 days ago

Hyderabad Secunderabad Telangana, India beBeeDataEngineering Full time
Job Description:

A Data Engineer with our team will work on designing and implementing end-to-end data pipelines, ensuring seamless data flow, automation, and reliable scheduling.

Responsibilities:
  • Collaborate with cross-functional teams to understand data requirements and translate them into efficient and scalable data solutions.
  • Design and develop data pipelines encompassing data ingestion, transformation, storage, and delivery.
  • Utilize expertise in programming languages like Python to write clean, maintainable, and optimized code for data processing tasks.
  • Leverage experience in Apache Spark and PySpark to distribute data processing across clusters and handle large datasets efficiently.
  • Possess a deep understanding of SQL (Oracle/SQL Server) and NoSQL databases (Big Data) to manage and query data effectively at scale.
  • Design and implement data pipelines on cloud platforms like AWS, Azure, or Snowflake for scalability and cost-effectiveness, leveraging services like S3, Blob Storage, or Data Lake Storage.
  • Orchestrate data pipelines using tools like Airflow or similar solutions to ensure smooth data flow, automation, and reliable scheduling.
  • Build and maintain integrations with RESTful and SOAP web services to facilitate seamless data exchange between systems.
  • Monitor and troubleshoot data pipelines to ensure data quality, consistency, and timely delivery.
  • Champion best practices for data engineering and maintain a high standard of code documentation.
Requirements:
  • 8+ years of experience in data engineering with a proven track record of designing and implementing data pipelines.
  • Strong programming skills in Python with proficiency in libraries like Pandas, Spark, and PySpark for data manipulation and analysis.
  • In-depth knowledge of SQL (Oracle/SQL Server) and NoSQL databases (Big Data) for data storage and retrieval at scale.
  • Experience with data pipeline orchestration tools like Airflow or similar solutions.
  • Experience designing and implementing end-to-end data solutions, from data ingestion to consumption.
  • Familiarity with cloud platforms (AWS, Azure, Snowflake) for data storage, processing, and services.
  • Understanding of web service protocols (REST, SOAP) and experience building data integrations.
  • Excellent problem-solving and analytical skills with a passion for building efficient data infrastructures.
  • Effective communication and collaboration skills to work effectively with cross-functional teams.
Education:
  • Bachelors / Master's of Engineering in Computer Science, Statistics, Mathematics, or a related technical field.
  • 8+ years of Software industry experience.

  • Data Engineering Specialist

    2 days ago

    Hyderabad / Secunderabad, Telangana, India beBeeDataEngineering Full time

    Job Title: Data Engineering SpecialistWe are seeking a highly skilled Data Engineering Specialist to design, build and maintain data lake solutions for scientific data. This vital role will drive business decisions for Research by developing scalable and high-performance data engineering solutions for large scientific datasets.You will collaborate with...

  • Data Engineering Specialist

    2 days ago

    Hyderabad / Secunderabad, Telangana, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Data Engineering SpecialistDetailed job description - Key Responsibilities:Lead the development and implementation of efficient data engineering solutions with a strong focus on Python and SQL programming.Design and build scalable, cost-effective datasets that support business reporting and AI/ML exploration using AWS services.Collaborate with...

  • Advanced Data Engineering Specialist

    4 days ago

    Hyderabad / Secunderabad, Telangana, India beBeeDataEngineering Full time

    Job Title: Advanced Data Engineering SpecialistOverviewSeeking a seasoned data engineering professional with 5+ years of experience in designing and implementing supply chain analytics solutions.Key ResponsibilitiesDesign, develop, and deploy large-scale data pipelines to process and transform complex data sets.Collaborate with cross-functional teams to...

  • Data Integration Specialist

    2 days ago

    Hyderabad / Secunderabad, Telangana, India beBeeDataIntegration Full time

    Job Description">As a Data Integration Specialist, you will be responsible for designing, developing, and maintaining data integration workflows and ETL (Extract, Transform, Load) processes using Informatica tools. Your goal will be to ensure that data is accurately extracted from source systems, transformed according to business requirements, and loaded...

  • Data Security Specialist

    4 days ago

    Hyderabad / Secunderabad, Telangana, India beBeeDataSecurity Full time ₹ 9,00,000 - ₹ 12,00,000

    Job Title: Data Security Specialist We are seeking a highly skilled and experienced Data Security Specialist to join our team. As a key member of our organization, you will be responsible for ensuring the security and integrity of our data by monitoring PKWARE & Cyera data scanning and classification jobs for errors and failures.You will provide first-line...

  • Senior Data Engineer Role

    2 days ago

    Hyderabad / Secunderabad, Telangana, India beBeeDataEngineering Full time

    As a Data Engineering Specialist, you will play a pivotal role in shaping our data engineering strategies.Must-Have Skills:Hands-on experience with cutting-edge technologies like Databricks, PySpark, SparkSQL, Apache Spark, AWS, Python, SQL, and Scaled Agile methodologies.Expertise in workflow orchestration, performance tuning on big data processing.Deep...

  • Data Engineer Specialist

    21 hours ago

    Hyderabad, Telangana, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 25,40,000

    Data Engineering SpecialistWe are looking for a skilled data engineering specialist to join our team. As a key member of the data engineering team, you will play a crucial role in designing and implementing complex data pipelines and ETL/ELT processes to support analytical and operational needs.

  • Data Engineering Specialist

    1 day ago

    Hyderabad / Secunderabad, Telangana, India beBeeAzure Full time ₹ 15,00,000 - ₹ 20,00,000

    Job OpportunityWe are seeking a skilled professional to join our team as an Azure Data Engineer.Greetings from our organization. We have multiple opportunities available for this role, offering a chance to work on exciting projects and expand your expertise.The ideal candidate will possess 5+ years of experience in data engineering with a focus on Azure...

  • Senior Data Engineering Specialist

    5 hours ago

    Hyderabad / Secunderabad, Telangana, Mumbai, Kolkata, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 20,00,000

    Job DescriptionWe are seeking a highly skilled Senior Data Engineering Specialist to join our team. The successful candidate will be responsible for designing and developing data pipelines that leverage a combination of technologies, including Vector DB, Graph DB, Airflow, Spark, PySpark, Python, LangChain, AWS Functions, Redshift, and SSIS.The ideal...

  • Cloud Data Engineering Specialist

    4 days ago

    Hyderabad / Secunderabad, Telangana, Pune, Chennai, India beBeeDataExpert Full time

    Cloud Data Engineering ExpertLead the end-to-end implementation of a cloud-based data platform.Craft and document robust data models, pipelines, and transformations within a unified data environment.Collaborate with cross-functional teams to ensure seamless integration of diverse data sources.Serve as a technical specialist for cloud-based data components,...