Transmission Jail!

Make sure to adjust the network adressing to your schema. All commands should be entered row by row.

iocage create -n "transmission" -r 12.2-RELEASE ip4_addr="vnet0|10.0.0.98/24" defaultrouter="10.0.0.1" vnet="on" allow_raw_sockets="1" boot="on" allow_tun="1"

We need TrueNAS to allow us to use tun devices inside the jail, to do this we need to activate it and also add it to a pre init task in the web ui.
Log in to TrueNAS, go to "Tasks" then "Init/Shutdown Tasks", then "Add". 

devfs rule -s 4 add path 'tun*' unhide

To be able to run everything immediately, we need to activate it now by running the same command in the terminal.

Install Packages

iocage exec transmission pkg install -y bash unzip unrar transmission-daemon transmission-web openvpn ca_root_nss nano

Setup paths, directories and user

mkdir -p /mnt/Jailer/apps/transmission
mkdir -p /mnt/dumpdisk/torrents/{incomplete,complete}/{tv,movies}
iocage exec transmission mkdir -p /config/transmission-home
iocage exec transmission mkdir -p /mnt/torrents/{complete,incomplete}
iocage fstab -a transmission /mnt/Jailer/apps/transmission /config nullfs rw 0 0
iocage fstab -a transmission /mnt/dumpdisk/torrents/incomplete /mnt/torrents/incomplete nullfs rw 0 0
iocage fstab -a transmission /mnt/dumpdisk/torrents/complete /mnt/torrents/complete nullfs rw 0 0
iocage exec transmission chown -R transmission:transmission /config/transmission-home
iocage exec transmission "pw user add media -c media -u 8675309 -d /nonexistent -s /usr/bin/nologin"
iocage exec transmission "pw groupmod media -m transmission"
iocage exec transmission chown -R media:media /mnt/torrents

IPFW rules

iocage exec transmission nano /config/ipfw.rules

Paste this and correct the adress scheme to yours!

#!/bin/bash
ipfw -q -f flush
cmd="ipfw -q add"
vpn="tun0"
$cmd 00001 allow all from any to any via lo0
$cmd 00010 allow all from any to any via $vpn
$cmd 00101 allow all from me to 10.0.0.0/24 uid media
$cmd 00102 allow all from 10.0.0.0/24 to me uid media
$cmd 00103 deny all from any to any uid media

VPN

Place your VPN providers configuration file in /mnt/Jailer/apps/transmission/config/openvpn.conf

Example: iocage exec transmission nano /config/openvpn.conf

integrity.st as an example, some content removed, better to always get the latest and greatest from you specific supplier!

log '/var/log/openvpn.log'
# integrity v5

client
dev tun
proto udp

remote openvpn.integrity.st 1198

remote-random
resolv-retry infinite
reneg-sec 0
nobind
persist-key
persist-tun
verb 3
mute 10
auth-user-pass
explicit-exit-notify 2
cipher aes-256-cbc
auth SHA512
script-security 2
remote-cert-tls server

auth-user-pass /config/userpass.txt

key-direction 1
<ca>
-----BEGIN CERTIFICATE-----
CONTENT REMOVED!
-----END CERTIFICATE-----
</ca>
<tls-auth>
#
# 2048 bit OpenVPN static key
#
-----BEGIN OpenVPN Static key V1-----
CCONTENT REMOVED
-----END OpenVPN Static key V1-----
</tls-auth>
Credentials

This is only if you are using a service with credentials needed to be enterd.

iocage exec transmission nano /config/userpass.txt

You only need to add this.

USRNAME
PASSWORD

Lets setup and configure sysrc

iocage exec transmission "chown 0:0 /config/ipfw.rules"
iocage exec transmission "chmod 600 /config/ipfw.rules"
iocage exec transmission sysrc "firewall_enable=YES"
iocage exec transmission sysrc "firewall_type=filename"
iocage exec transmission sysrc "firewall_script=/config/ipfw.rules"
iocage exec transmission sysrc "openvpn_enable=YES"
iocage exec transmission sysrc "openvpn_dir=/config"
iocage exec transmission sysrc "openvpn_configfile=/config/openvpn.conf"
iocage exec transmission sysrc "transmission_enable=YES"
iocage exec transmission sysrc "transmission_conf_dir=/config/transmission-home"
iocage exec transmission sysrc 'transmission_user=media'
iocage exec transmission sysrc 'transmission_group=media'
iocage exec transmission service ipfw start
iocage exec transmission service openvpn start
iocage exec transmission service transmission start

Finetuning transmission

iocage exec transmission service transmission stop

We also need to get access to the service, either only you or all in local network.

iocage exec transmission nano /config/transmission-home/settings.json

Easy way, disable rpc whitelist.

"rpc-whitelist-enabled": false,

Other way, enable rpc whitelist and add your ip.

"rpc-whitelist-enabled": true,
"rpc-whitelist": "127.0.0.1,10.0.0.10",

Also, for more control i like to add the downloaded (finished) in a "complete" folder and vice versa.
To do this we adjust some more in the config.

Change

"download-dir": "/usr/local/etc/transmission/home/Downloads",
"incomplete-dir": "//Downloads",
"incomplete-dir-enabled": false,
"ratio-limit-enabled": false,

To

"download-dir": "/mnt/torrents/complete",
"incomplete-dir": "/mnt/torrents/incomplete",
"incomplete-dir-enabled": true,
"ratio-limit-enabled": true,

https://github.com/amussey/FreeNAS-Transmission-OpenVPN/issues/3#issuecomment-252860898

In the freenas control panel under System>Tunables add a new tunable
kern.ipc.maxsockbuf
5242880
type: sysctl

and

net.inet.udp.recvspace
4194304
type: sysctl

save and reboot