Senior Data Engineer
2 days ago
Role Overview
As a Big Data Engineer at BrightEdge, you will design, build, and maintain high-performance data pipelines that process terabytes of data. You'll work on optimizing our existing systems, identifying and resolving performance bottlenecks, and implementing solutions that improve the overall efficiency of our platform. This role is critical in ensuring our data infrastructure can handle increasing volumes of data while maintaining exceptional performance standards.
Key Responsibilities
● Design and implement scalable batch processing systems using Python and big data technologies
● Optimize database performance, focusing on slow-running queries and latency improvements
● Use Python profilers and performance monitoring tools to identify bottlenecks
● Reduce P95 and P99 latency metrics across our data platform
● Build efficient ETL pipelines that can handle large-scale data processing
● Collaborate with data scientists and product teams to understand data requirements
● Monitor and troubleshoot data pipeline issues in production
● Implement data quality checks and validation mechanisms
● Document data architecture and engineering processes
● Stay current with emerging big data technologies and best practices
Qualifications Required
● Bachelor's degree in Computer Science, Engineering, or related technical field
● 4+ years of experience in data engineering roles
● Strong Python programming skills with focus on data processing libraries
● Experience with big data technologies (Spark, Hadoop, etc.)
● Proven experience optimizing database performance (SQL or NoSQL)
● Knowledge of data pipeline orchestration tools (Airflow, Luigi, etc.)
● Understanding of performance optimization techniques and profiling tools
● Experience with cloud platforms (AWS, GCP, or Azure)
● Ability to translate business requirements into technical specifications
● Ability to work in UK timezone (12 PM IST to 9:30 PM IST)
Preferred
● Master's degree in Computer Science or related field
● Experience with SEO data or web crawling systems
● Experience with Clickhouse Database
● Knowledge of distributed systems and microservices architecture
● Familiarity with container orchestration (Kubernetes, Docker)
● Experience with real-time data processing
● Contributions to open-source projects
● Experience with machine learning operations Skills and Abilities
● Strong analytical thinking and problem-solving skills
● Excellent attention to detail and commitment to data quality
● Ability to work effectively in a collaborative team environment
● Good communication skills to explain complex technical concepts
● Self-motivated with the ability to work independently
● Passion for performance optimization and efficiency
-
Senior Data Engineer
3 weeks ago
India Aptus Data Labs Full timeJob Title: Senior Data EngineerLocation: RemoteExperience: 4+Employment Type: Full-TimeNotice- Looking for immediate joiners or who can join within 15-20 days.About the RoleAptus Data Labs is looking for a talented and proactive Senior Data Engineer to help build the backbone of our enterprise data and AI initiatives. You'll work on modern data lake...
-
Senior Data Engineer
4 weeks ago
India Asenium Consulting Full timeFor one of my customers, we are looking for a Senior Data Engineer.Location: India (100% remote)Duration: 6+ months (renewable)Description: We are looking for a Senior Data Engineer with strong experience in SQL, Talend, and cloud platforms such as Google Cloud (Big Query) and Microsoft Azure.You will be responsible for designing and managing ETL pipelines,...
-
Senior Engineer, Data Engineering
3 hours ago
India Collabera Full timeSenior Data Engineer Fully Remote Shift: US Time Zone (Night Shift)Financial Institution (FI) - Credit Union Collabera is seeking a seasoned Senior Data Engineer for its leading financial customer to design and optimize data solutions that power our analytics, reporting, and Member 360 initiatives. This role is ideal for someone with deep expertise in ETL...
-
Senior Data Engineer
2 weeks ago
India 3Pillar Global Full timeJob DescriptionEmbark on an exciting journey into the realm of software development with 3Pillar We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making.As a Senior Data Engineer, you will hold...
-
Senior Data Engineer
4 weeks ago
India Black Piano Full timeOverviewAs a Senior Data Engineer, you'll play a crucial role in storing, processing, modelling, and applying data science to make data and insights available for analytics and business intelligence (BI) systems. This position offers a unique opportunity to work with cutting-edge products and world-class clients in a remote-first environment.Key...
-
Data Engineer
4 weeks ago
India Indsafri Full timeHiring Now: Senior Data Engineers (Banking Domain – 2 Positions)Location: RemoteExperience: 7+ Years | Banking sector experience preferred Type: Contract About the OpportunityWe're looking for highly skilled Senior Data Engineers to join a mission-critical banking project. This is your chance to work on cutting-edge data pipelines, streaming platforms,...
-
Data Engineer
2 days ago
India Insight Global Full time100% Remote Data EngineerRequired Skills & Experience5+ years of experience as a Data EngineerExpertise in Python, Pyspark and SQLHands-on Python/PySpark coding experienceStrong Big Data experience, GCP preferredTelemetry experienceStreaming experience with analytics and data processingBusiness analytics experience and working directly with business partners...
-
Senior Data Engineer – GCP
4 weeks ago
India Xebia Full timeWe're Hiring: Senior Data Engineer – GCP | Databricks | E-commerce Domain Work Locations: Chennai | Bangalore | Hyderabad | Gurugram | Jaipur | Pune | Bhopal Experience: 7–8+ Years Shift Timing: 2 PM – 10 PM IST Work Mode: Hybrid – 3 days/week from office Only Immediate Joiners (0–15 Days Notice)Are you a seasoned Data Engineer passionate about...
-
Senior Engineer, Data Engineering
3 hours ago
India BrightEdge Full timeBig Data Engineer - BrightEdge About Us BrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results. Our platform processes massive amounts of data to provide actionable insights to our clients. We're looking for a talented Big Data Engineer to join our Professional Services to help us...
-
TAVS Senior Data Engineer(Location: Pune)
4 weeks ago
India DigiHelic Solutions Pvt. Ltd. Full timeTAVS Senior Data EngineerLocation: PuneExperience: 12 Years1. Proficiency in PostgreSQL, including database design, optimization, and management.2. Extensive experience with Databricks for data processing and analytics, and/or any other Apache Spark distribution.3. Strong programming skills in Python, especially with libraries such as Pandas and NumPy.4....
