上海招聘

职位： 数据工程师

部门： Content

工作地点：上海

汇报线： Senior Manager of Content Team, Shanghai

岗位职责：

1. 负责公司大数据的爬虫与清洗工作

2. 设计爬虫策略和防屏蔽规则，提升网页抓取的效率和质量

3. 负责大规模文本、图像的抓取、抽取、去重、分类，垃圾过滤，质量识别等

4. 研究各种目标网站的形态，互联网特征挖掘，发现它们的特点和规律

5. 设计和维护内部使用的数据库

任职要求：

1. 本科及以上学历

2. 了解Mysql等常用数据库

3. 熟练掌握一种爬虫框架，例如Scrapy或者Selenium

4. 熟悉常见的反爬虫机制解决办法

5. 熟悉Internet基本协议（如TCP/IP. HTTP等）

6. 熟悉缓存、消息机制、并发框架，异步框架等

7. 熟悉HTML/DOM/XPATH/CSS，对DOM有一定的功底能够熟练使用Jsoup等进行分析网页，有模拟登录、模拟浏览器、APP抓取经验者优先

8. 一年以上网络抓取及数据挖掘的经验，应届生具备Python项目经验亦可

9. 谦虚勤奋，有良好的时间管理能力，以结果为导向

加分技能：

1. 有一定的PDF数据自动化处理相关知识，优先考虑

2. 有网络应用开发的经验

3. 有用Python、C#或Delphi开发UI应用程序的经验

Job Title: Data Operations Developer

Department: Content

Work Location: Shanghai, China

Reports to: Senior Manager of Content Team, Shanghai

Responsibilities:

1. Craw and clean the company’s big data.

2. Design crawler strategies and anti-blocking rules to improve the efficiency and quality of web crawing.

3. Large-scale text, image data capture, extraction, deduplication, classification, garbage filtering, quality recognition, etc.

4. Research the morphology of various target websites, mining Internet characteristics, and discover their characteristics and laws.

5. Deploy and maintain internally used databases.

Requirements:

1. Bachelor Degree above.

2. Familiar with Mysql and other common databases.

3. Proficiency in a crawler framework, such as Scrapy or Selenium.

4. Familiar with common anti-crawler mechanism solutions.

5. Familiar with basic Internet protocols (such as TCP/IP, HTTP, etc.).

6. Familiar with caching, message mechanism, concurrency framework, asynchronous framework, etc.

7. Familiar with HTML/DOM/XPATH/CSS. Basic knowledge of DOM and proficiency in analyzing web pages by Jsoup, etc. Experience in simulated login, simulation of browser and APP fetching is preferred.

8. 1+ year above working experience or project experience in web crawling and data

mining.

9. Diligent and humble. Good time management skill, and result oriented behavior mode.

The following skills is a plus:

1. Knowledge of PDF data automation.

2. Experience of developing web applications.

3. Experience of developing applications with UI in python, C# or Delphi.

Altium Limited (ASX: ALU) is an Australian multinational software corporation that focuses on electronics design systems for 3D PCB design and embedded system development. Altium products are found everywhere from world leading electronic design teams to the grassroots electronic design community. With a unique range of technologies, Altium helps organisations and design communities to innovate, collaborate and create connected products while remaining on‐time and on‐budget. Products provided are Altium Designer？, Altium Vault？, CircuitStudio？, CircuitMaker？, the TASKING？ range of embedded software compilers and Octopart？ the search engine for electronic components and industrial products. Founded in 1985, Altium has offices worldwide, with US locations in San Diego , New York and Boston, European locations in Munich, Karlsruhe, Amersfoort, Kiev and Zug and Asia‐Pacific locations in Shanghai, Tokyo and Sydney. For more information, visit **************. You can also follow and engage with Altium via Facebook, Twitter and YouTube.
 
You can refer to ************** for more information about Altium.
 
Altium有限公司(ASX:ALU)是业界领先的电子产品开发解决方案开发商，总部位于美国加州La Jolla, 已于1999年在澳大利亚股票交易所IPO上市。目前亚太总部位于上海，在深圳、北京等地拥有分公司，并在全球拥有28处分公司和办事处。
Altium致力于把电子产品开发过程中涉及到的不同设计规程统一到单一的设计系统中。 Altium产品保证所有电子工程人员、设计人员、开发人员及其公司无限度地利用新兴设计技术，更快更容易地向市场推出更智能的产品。其主要产品包括Altium Designer（Protel）、P-CAD、Nexar等，面向电子产品设计、开发主流市场。