mirror of http://git.simp.i2p/simp/i2pnews.git
78 lines
3.7 KiB
Markdown
Executable File
78 lines
3.7 KiB
Markdown
Executable File
## I2pnews webapp and bot
|
|
This is a both the webapp at i2pnews.simp.i2p and the bot present in #i2p-news. It follows the feed list from `rsslist.txt` and periodically updates all feeds using aiohttp, asyncio and feedparser.
|
|
|
|
If a feed has been updated, it saves the last x amount of posts to a txt file under `/static/rssfeed` for each category. Each category is color-coded by the bot and given a seperate page and consolidated rss feed with FeedGenerator. Post times are changed to UTC.
|
|
|
|
The webapp displays a page that shows the ping times and last seen for each feed, giving a text list of alive feeds and all feeds given in json format at /json.
|
|
|
|
### Todo
|
|
The bot was built first, and the webapp built around the irc bot. It'd be better if they were seperate with the webapp functioning independently and puppeting the ircbot through api, i think.
|
|
|
|
[DONE] **Cap per feed per update:** simple max of x number new posts on the irc bot so its not a wall of text. Feeds are capped at 5 in IRC for the live bot now.
|
|
|
|
[WIP] **I2p urls:** The feed links are all over the place, some point to clearnet, or localhost, or not the post. Have the links point to valid i2p only links. Localhost is corrected and some site specific corrections.
|
|
|
|
[X] **Cleanup:** a lot has been copied from other projects, it's messy
|
|
|
|
[X] **Optimization:** this could be leaner, it runs around 85MB now. If we keep the feeds in memory it can run faster, it's not doing that now.
|
|
|
|
### Install
|
|
If for some reason you want to install this, that's probably a mistake on your part but, clone the repo using your i2p http proxy, cd into it, then make a python venv.
|
|
|
|
Activate the environment, then run `pip3 install -r requirements.txt`
|
|
|
|
Something like this should work:
|
|
|
|
git clone --config http.proxy=127.0.0.1:4444 http://git.simp.i2p/simp/i2pnews.git && cd i2pnews && python3 -m venv ./ && cd bin && source activate && cd .. && pip3 install -r requirements.txt
|
|
|
|
After requirements are installed, while in your venv running `python3 wsgi.py` should work, which will run gunicorn. Running `python3 app.py` will just run the flask app withou gunicorn. Useful for debugging not much else.
|
|
|
|
Recommended to run the wsgi.py through nginx or apache, example config with nginx pointing i2p tunnel to 5050:
|
|
|
|
server {
|
|
listen 5050;
|
|
server_name somedomain.i2p;
|
|
server_tokens off;
|
|
if ($request_method !~ ^(GET|POST)$ ) {
|
|
return 444;
|
|
}
|
|
if ($http_user_agent ~* LWP::Simple|BBBike|wget) {
|
|
return 444;
|
|
}
|
|
location / {
|
|
proxy_hide_header Server;
|
|
proxy_pass http://unix:/path/to/newsbot/newsbot.sock;
|
|
}
|
|
}
|
|
|
|
Listens in on the socket on `/path/to/newsbot/newsbot.sock` so make sure you change that to the full directory.
|
|
|
|
### Settings
|
|
Most settings are in website.ini file
|
|
|
|
[proxy]
|
|
hostname = 127.0.0.1
|
|
http_tunnel = 4444 - setup a seperate http tunnel under java to close after idling x minutes and open with new keys on activity
|
|
|
|
[rss]
|
|
news_bot = botname
|
|
news_pass = botpassword
|
|
news_irc_port = 6667
|
|
news_rss_flask_port = 5001 - port to serve flask. not used if using wsgi like gunicorn
|
|
news_rss_ah = addresshelperlink
|
|
news_email = nobody@mail.i2p
|
|
auth_users = nobody, anothernick - irc nicks authed to do $admin commands
|
|
channel = #i2p-news - original channel to join, changed with $admin main_channel #channel1 #channel2 or $admin join/part #channel
|
|
primary = nobody - sends msgs to this nicks memoserv on admin commands
|
|
gunicorn_port = 5000 - for running production with wsgi
|
|
|
|
### Add feeds
|
|
feed list is `rsslist.txt`, format is url category
|
|
categories: news, blog, forum, developer, torrents
|
|
example:
|
|
|
|
http://zzz.i2p/topics.rss forum
|
|
|
|
Then in IRC do:
|
|
|
|
$admin refresh |