i2music/README.md

# i2music

This is a js-free music player. It uses iframes and css for interactivity and passes what's needed to the backend. Allowing use of cookies extends functionality.

Live instance at <a href="http://music.simp.i2p" target="_blank">music.simp.i2p</a>

### Features

- Minimal audio player using only html/css
- Can load balance streams to a number of b32s, see number of listeners
- Optional separate stream server
- Add music files to directory, they function as playlists
- Automatic encode of all audio files to desired bitrate, compression of images
- Torrent packing for playlists by hashing and sending to torrent client 
- Tracker scraping for peer counts
- Basic search
- Limited controls: forward/back track controls, stop, resume last listened to track
- User settings: 4 themes, repeat, delay
- Manual ordering of tracks, otherwise ordered using natsort

Features that need to remember something about the user need cookies to work: resume, all settings.

Otherwise default values are used or it just ignores it.

The `updateplaylists.py` script will compress all found tracks to opus and change image files to progressive jpeg. This should make it easy to add tracks and convert everything to work better with low bandwidth. If the webapp is running it will pass on new track info.

This script will also pack a playlist for torrenting, by hashing the contents of the directory and creating a symlink to a torrent directory. It then copies the .torrent file there.

### To be added

- Track metadata

# Install Guide

This will install the webapp stack and all dependencies.
wsgi.py runs gunicorn, which runs the flask app. This passes either a socket file or a port to the webserver. It's recommended to run nginx with gunicorn, generally, though other webservers can be used. For streaming, we want nginx to be doing the heavy lifting.

## Step 1: Make a user for the install

Makes a "music" user:

	sudo useradd music && sudo mkdir /home/music && sudo chown music:music /home/music && sudo usermod -aG sudo music && sudo passwd -d music && su music
	
Make sure ffmpeg is installed. Run:

	ffmpeg -version
	
If not installed run (check how to install for your OS):

	sudo apt update && sudo apt install ffmpeg

## Step 2: Download and install dependencies

Next, cd to the home directory of the "music" user:

	cd /home/music
	
Download and run the install script:

	curl --retry 5 -x http://127.0.0.1:4444 -O http://git.simp.i2p/simp/i2music/raw/branch/main/install.py && python3 install.py

Run this to test that it's running:

	cd i2music && source env/bin/activate && python3 wsgi.py
	
It should be running on ports 5000 and 5005. 

### ONLY if that doesn't work, otherwise skip this part - Manual install

Clone the repo using your i2p http proxy, cd into it, then make a python venv. 

Activate the environment, then run `pip3 install -r requirements.txt`

Something like this should work:

	git clone --config http.proxy=127.0.0.1:4444 http://git.simp.i2p/simp/i2music.git && cd i2music && python3 -m venv ./ && cd bin && source activate && cd .. && pip3 install -r requirements.txt

After requirements are installed, make sure your venv is activated:
	
	source env/bin/activate

Running `python3 wsgi.py` should work, which will run gunicorn. Running `python3 app.py` will just run the flask app withou gunicorn. Useful for debugging. Do not run the flask app alone without wsgi! 

## Step 3 - Edit config

in the /i2music directory, do 

	nano config.txt
	
Make sure to change:

	secret_key = <change_it>
	
Worth a look:
	
- site_title 
- options under [footer]
- bitrate 
- if you want to get peer counts for hashed torrents, change get_peercounts to True

The settings section has descriptions on the other options. To take effect the  app needs to be restarted.

## Step 4 - Run the app

Running as a service will keep it running under the "music" user we created. 

### As a service (systemd) with socket file

This config will work for systemd linux. Run:

	sudo nano /etc/systemd/system/i2music.service
	
Then paste the following:

	[Unit]
	Description=Gunicorn instance to serve music
	After=network.target

	[Service]
	User=music
	Group=www-data
	WorkingDirectory=/home/music/i2music
	Environment="PATH=/home/music/i2music/bin"
	ExecStart=/home/music/i2music/bin/gunicorn --workers 1 --bind unix:eepsite.sock -m 007 wsgi:app

	[Install]
	WantedBy=multi-user.target

Make sure the paths and user are correct. Then do ctrl+s to save, ctrl+x to exit.

Now we enable and start the service:

	sudo systemctl daemon-reload && sudo systemctl enable i2music && sudo systemctl start i2music

You should see a file "eepsite.sock" in the /i2music directory when running.

### Run the wsgi.py directly 

You can run just the wsgi.py directly, passing the socket file or port to the webserver. It's recommended to use the socket file option over the port if possible.

In the /i2music directory activate the venv:

	source env/bin/activate

To get a .sock file, run:

	/path/to/i2music/bin/gunicorn --workers 1 --bind unix:eepsite.sock -m 007 wsgi:app

OR, to get just the port, run:

	python3 wsgi.py

## Step 5 - Setup Nginx

This will pass the eepsite.sock file through nginx, with nginx listening on port 5050 (what we want to give the i2p tunnel).

Change "server_name" and make sure the path in "proxy_pass" is right.

Go the the nginx directory (path may be different):

	cd /etc/nginx

Make a new site in sites-enabled

	sudo nano /etc/nginx/sites-available/i2music
	
Then paste the config, change server_name somedomain.i2p and make sure the path in proxy_pass is right:

	server {
    	listen 5050;
    	server_name somedomain.i2p;
    	server_tokens off;
    	if ($request_method !~ ^(GET|POST)$ ) {
        	return 444;
    	}
    	if ($http_user_agent ~* LWP::Simple|BBBike|wget) {
        	return 444;
    	}
		keepalive_timeout 320;
		proxy_buffering on;
		proxy_buffer_size 256k;
		proxy_busy_buffers_size 264k;     
		proxy_buffers 128 8k;
		gzip on;
		gzip_comp_level 6;
		gzip_buffers 32 4k;
		gzip_types text/plain text/css application/json image/gif image/jpeg image/png image/webp application/javascript text/xml application/xml application/xml+rss application/atom+xml audio/ogg audio/mp3 font/truetype font/opentype image/svg+xml
		reset_timedout_connection on;
		client_body_timeout 20;
		send_timeout 10;
    	location / {
        	proxy_hide_header Server;
        	proxy_pass http://unix:/home/music/i2music/eepsite.sock;
    	}
	}

ctrl+s / ctrl+x to save and exit.

Now make symlink to sites-enabled.

	sudo ln -s /etc/nginx/sites-available/i2music /etc/nginx/sites-enabled/i2music

Reload Nginx so it sees the new config:

	sudo systemctl reload nginx

## Step 6 - Setup i2p tunnels

From here you only need to pass the port 5050 from nginx to i2p. Don't use port 5005 (gunicorn) and never run the flask port anywhere other than locally, as its unsafe (5000)

In java, make a new http server tunnel. Make sure to save the private key.

Start with 4-6 tunnels, you can opt to reduce tunnels to conserve resources.

There are some custom tunnel options you can add under the next section that you can experiment with, but they seem to optimize the tunnel for streaming and may be at the cost of loading html pages.

To optimize both, you can consider offloading the music stream from the webapp tunnels.

# Increase performance with separate stream tunnels (optional)

This requires more setup, but will give a more performative stream and scalability. It will also let us optimize a set of tunnels for streaming, and save the ones for the app serving web pages.

You can reduce the tunnel count on your main app, 4 is likely sufficient for inbound for good reliability, and use "reduce tunnels to conserve resources" option in java to allow the stream and app tunnels to die down to 2 when not in use.

We will run "wsgi_fileserve.py" alongside the normal "wsgi.py", which will start another gunicorn instance on another port, "gunicorn_stream_port = 5006" under the settings block. All this server instance can do is serve files with GET requests, all others will be dropped.

## Streaming server systemd service config

This mirrors the setup from step 4.

Run:

	sudo nano /etc/systemd/system/i2musicstream.service
	
Then paste the following:

	[Unit]
	Description=Gunicorn instance to serve music streams
	After=network.target

	[Service]
	User=music
	Group=www-data
	WorkingDirectory=/home/music/i2music
	Environment="PATH=/home/music/i2music/bin"
	ExecStart=/home/music/i2music/bin/gunicorn --workers 1 --bind unix:stream.sock -m 007 wsgi_fileserve:app

	[Install]
	WantedBy=multi-user.target

Enable and start the service:

	sudo systemctl daemon-reload && sudo systemctl enable musicstream && sudo systemctl start musicstream

# Nginx configs for streaming server and webapp

We will have one optimized for streaming, and one for the app

## Streaming server Nginx config

Go the the nginx directory (path may be different):

	cd /etc/nginx

Make a new site in "sites-enabled"

	sudo nano /etc/nginx/sites-available/musicstream
	
Then paste the config, change "server_name" and make sure the path in "proxy_pass" is right:

	server {
    	listen 5051;
    	server_name somedomain.i2p;
    	server_tokens off;
    	if ($request_method !~ ^(GET)$ ) {
        	return 444;
    	}
    	if ($http_user_agent ~* LWP::Simple|BBBike|wget) {
        	return 444;
    	}
		keepalive_timeout 320;
		proxy_buffering on;
		proxy_buffer_size 256k;
		proxy_busy_buffers_size 264k;     
		proxy_buffers 128 8k;
		gzip on;
		gzip_comp_level 6;
		gzip_buffers 32 4k;
		gzip_types text/plain text/css application/json image/gif image/jpeg image/png image/webp application/javascript text/xml application/xml application/xml+rss application/atom+xml audio/ogg audio/mp3 font/truetype font/opentype image/svg+xml
		reset_timedout_connection on;
		client_body_timeout 20;
		send_timeout 10;
    	location / {
        	proxy_hide_header Server;
        	proxy_pass http://unix:/home/music/i2music/stream.sock;
    	}
	}
	
## Webapp-only Nginx config update

We can take things out that are geared at optimizing streaming. 

Edit the i2music config made earlier

	sudo nano /etc/nginx/sites-available/i2music
	
Remove the parts between "keepalive_timeout 320;" and "location / {" so it looks like this:

	server {
    	listen 5050;
    	server_name somedomain.i2p;
    	server_tokens off;
    	if ($request_method !~ ^(GET|POST)$ ) {
        	return 444;
    	}
    	if ($http_user_agent ~* LWP::Simple|BBBike|wget) {
        	return 444;
    	}
		keepalive_timeout 320;
    	location / {
        	proxy_hide_header Server;
        	proxy_pass http://unix:/home/music/i2music/eepsite.sock;
    	}
	}

ctrl+s / ctrl+x to save and exit.


Reload Nginx so it sees the new config:

	sudo systemctl reload nginx
	
	
## Make i2p http server tunnels for streaming only

Make a new http server tunnel, listening for port 5051 (based on nginx config). In java, try starting with 4 tunnels for up/down. Enable "reduce tunnels to conserve resources" after 5 mins to 2 tunnels. Save the key file, start automatically when router starts. 

Under custom options, paste this and adjust to your liking:

	i2cp.fastReceive=true i2cp.gzip=true i2p.streaming.bufferSize=256k i2p.streaming.connectTimeout=20*1000 i2p.streaming.inactivityTimeout=600*1000 i2p.streaming.limitAction=http

Once you have one tunnel made up, copy it so you have 2 or more sets of streaming tunnels, all going to port 5051. You can make as many of these tunnels as needed with different b32's, using identical settings.


Then copy each of the server b32's, add 1 per line, tabbed. Found in the config.txt file under the "b32streams" block, like this:

    [b32streams]
    b32s =
        hvyaj6xegbsluf65ob2rdw2yf25me7kavmy2o3crwtijf4yb4zwa.b32.i2p
        gkbtx3a2s3pryryiagso7e5rehcbccphyvdm4y2nxops77os56nq.b32.i2p

After restarting the main webapp, it will keep track of the number of listeners using each b32 and try to use the least crowded one. A user will continue using the same stream server for the session.

# Playlist update script: updateplaylists.py

Note: this script need ffmpeg to work. Make sure that's installed.

After install you can add music of various raw formats to a directory under `/static/playlists`

Each directory added under "playlists" will automatically be added as a playlist. You can prevent certain playlists from having their audio compressed by adding to the exclude list:

    [playlists]
    exclude = playlist1, playlist2
 
For images, you can add a cover image. Make sure it's called "cover.xxx" (any major image extension). If you want an image for each track, make the music track and the image file have the same title. Like "song1.wav" and "song1.jpg"

Then, while in your venv run:

	python3 updateplaylists.py
	
This will compress all found music tracks to opus and the given bitrate, and change all images found to progressive jpeg. This should enhance load times.

96k offers great quality with opus and streaming works well in i2p. The majority of tracks on music.simp.i2p are encoded at 96k.
64k is good quality, streaming may be more reliable than 96k.
32k is perfect for voice only. The podcasts/youtube audio on music.simp.i2p are encoded at 32k.

	[audio_settings]
	bitrate = 96k

The script won't touch images and music files that have already been converted. BUT IT WILL REPLACE those files, so make sure you have the originals somewhere else.

# Torrent packing with updateplaylists.py 

The options for this are under the "torrent" section.

	[torrent]
	make_playlist_torrents = True
	wait_before_adding_torrent = 10
	get_peercounts = True
	scraper_hostname = 127.0.0.1
	scraper_http_proxy = 4444
	scrape_interval = 1200
	torrent_directory =

To enable hashing of torrents, `make_playlist_torrents` should be True. This will automatically make a torrent of each playlist using the trackers in `tracker_list`.

`get_peercounts` will enable the tracker scraping feature, which will show peercounts of the trackers in your `tracker_list`. This will use your i2p http proxy to work, so it must be set. The default is 127.0.0.1:4444. Scraping will occur based on the `scrape_interval`, which is given in seconds. 20mins to an hr or even more is fine, it likely won't do much good to have the interval too short other than get your tunnel ratelimited.

`torrent_directory` is the full path to a watched torrent directory. This could be the same directory snark (or biglybt/qbittorrent) use. To avoid having 2 copies of of everything, it will make a symlink for each data path. Then it will copy the generated .torrent there. qbittorrent and biglybt can start the torrent right away when they see it, snark will mark it as "completed" and will need to be started manually (there's a start feature in the multisnark tool in [tuckit](http://git.simp.i2p/simp/TuckIt))

With all features enabled, the updateplaylists.py script will compress the contents of enabled playlist directories, hash them as torrents, send them to your torrent clients watch folder, and save the playlist data as `data.json`

# How it works

When the webapp is ran it loads the data.json file that was created from running `python3 updateplaylists.py`

For each music file it gets the track length, name, looks for a matching image, and if not found falls back to a playlist "cover" image, and if none found there falls back to "nocover.jpeg"

These details are kept in memory to determine track controls (next/previous song) and how long to wait before loading the next track. This is done with "meta refresh" in html, passing the time in seconds to each track, which is its own page in an iframe.

There are 4 iframes, left is for playlist view, center is the audio player, right top is for settings, right bottom is for info. Using target="framename" we can get some limited interactivity this way, relying on backend.

# Track ordering

Tracks are ordered using natsort, which sorts a-z/numerically.

To manually order tracks, name them with a "xxx." in front of the title:

	1. First track
	2. Second track
	3. Third track...

The number in front of the track name won't be shown in the player.

# Browser support

This relies entirely on browser suport for playing audio and autoplay especially. Major browsers should all support wav/mp3/ogg(opus), but some may block autoplay. The user may need to enable it to work right.

# Settings
Most settings are in config.txt file  

    [settings]
    hostname = 127.0.0.1 #usually leave as is
    site_port = 5000 #ONLY for flask and debugging
    stream_port = 5001 #only used if running separate stream server (optional)
    gunicorn_port = 5005 #this is the gunicorn port
    gunicorn_stream_port = 5006 #only used if running separate stream server (optional)
    secret_key = <secret_key> #change this
    changelog = changelog.txt 
    log_file_location = log.txt 
    
	[appearance]
	site_title = Test player #shows on browser tab
	default_theme = darkgreen
	theme_list = darkgreen:#02AE16FF, pink:#FF00E1FF, blue:#00a2ff, red:#b80000 #themename:#hexcolor that shows up in the theme selecter icon. don't touch unless you want to disable a theme or add your own. 
	image_resize_kb = 150 #target size for resizing found images

	[footer]
	left_footer = <a href="http://skank.i2p" target="_blank"> Running on I2P+</a> #footer, left side
	right_footer = <a href="http://git.simp.i2p/simp/i2music" target="_blank">GIT</a> #footer, right side
	ah = <address helper link> #change to address helper link

    [audio_settings]
    bitrate = 96k #used for updateplaylists.py
    default_image = nocover.jpeg #fallback image if no images found
    default_delay = 1 #seconds for additional delay on loading next track

    [playlists]
    subpath = playlists # /static/playlists/ for your directories of playlists
    exclude = #audio files in these playlists won't be compressed when running updateplaylists.py
    
    [b32streams]
    b32s = #leave this empty unless you want to load balance streams over other b32's, or offload streaming to another b32. One per line, tabbed on each like tracker_list.

	[torrent]
	make_playlist_torrents = True #if true, a .torrent file will be generated of each playlist
	wait_before_adding_torrent = 60 #seconds to wait. if a torrent of the same name is different (updating a playlist), it will delete and wait for torrent client to see the change before adding the new torrent.
	torrent_directory = add a directory watched by a torrent client like snark
	get_peercounts = False #if true, trackers from the tracker_list will be scraped.
	scraper_hostname = 127.0.0.1 #hostname for i2p http proxy
	scraper_http_proxy = 4444 #port for i2p http proxy
	scrape_interval = 1200 #how often (seconds) to scrape all trackers

    [trackers]
    tracker_list = #list is used for hashing torrents and scraping trackers
        http://opentracker.simp.i2p/a
        http://opentracker.dg2.i2p/a
        http://opentracker.skank.i2p/a
        http://opentracker.r4sas.i2p/a
        http://opentracker.eeptorrent.i2p/a
        http://omitracker.i2p/announce.php
        http://6kw6voy3v5jzmkbg4i3rlqjysre4msgarpkpme6mt5u2jw33nffa.b32.i2p/announce
        
Postmans tracker was ommited because it requires manual upload where the opentrackers don't. If you're planning to do that add `http://tracker2.postman.i2p/announce.php` to the top of the list.