Skip to content
This repository was archived by the owner on Jun 10, 2021. It is now read-only.

hi20160616/fetcher

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

42 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Fetcher

Fetch news from urls.

Intro

Fetch depth: 1

Development Tips

There files need to be modified while add a new site:

  • main.go: add entrance url
  • links.go -> SetLinks(): add case about target urls feature regex, eg: if url must have about, param 2 is .*?about.*
  • site/newsite: copy files from sibling folder, then develop and pass the test.
  • post.go -> TreatPost(): add case for new site domain.

About

A Crawler to fetch news.

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors

Languages