成都招聘

Job Responsibilities

We are looking for outstanding Big Data folks to join us to build data set for model training, risk detection and production issues investigation, to address real business needs. If you are passionate about technology, no matter big data or frontend, love analytics, welcome to join! Team members in AOS are mostly full-stack engineers with master degrees from top universities (Fudan, SJTU, Tongji, etc.)

Key Day-to-Day Responsibilities includes:

Develop & maintain the ETL batch to manipulate the model result data (live logging by Kafka) on Hadoop platform (HDFS), the data is semi-structured and huge (~5TB daily), combine analytical data set with the logging data for model training, investigation & monitoring purpose.

Support, trouble-shooting & educate the users to access the data in Hadoop data mart, by developing toolkits (both web and batch process) to provide best-class data services to end users and applications for integration. Keep improving the user experience according to the feedback; automate data process pipeline with innovative way, to reduce the manual effort and cost of ownership.

Requirements

Bachelor degree or equivalent in engineering related background, with at least 2 years’ experience (Master for at least 1 year) in data processing area.

Experience in working under Unix/Linux, solid Shell scripting capability.

Master at least one of the programming languages (Java, Scala, Python, etc.), better having experience in data processing area.

Web frontend experience is a big preference.

Experience in Hadoop MapReduce, experience on Pig, SPARK etc. batch processing is a big plus

Knowledge of statistics, data mining or predictive modeling is plus

Experience in job scheduler (e.g. UC4, Airflow, etc.) is plus

Key Competencies Needed:

Willing to work as DevOps, customer oriented

Be proactive to learn new things

Good expression with English in write & read

Strong problem-solving skills to resolve issue quickly and smartly.

分享

微信 邮件