"arguing that you don’t care about the right to privacy because you have nothing to hide is no different than saying you don’t care about free speech because you have nothing to say."
For those unfamiliar: SMSSecure (now called Silence) is a text messaging application that supports encryption. It has exceptional integration on your phone and will handle your non-encrypted friends acquaintances texts as well. https://smssecure.org
I recommend using fdroid to install it. F-droid is a Free Software repository for Android. https://f-droid.org
UPDATE: You can now export an encrypted copy of Silence. This export includes your encryption keys. From your old phones Silence “app”lication, under the three horizontal “…” -> “Import / Export” -> “Export encrypted backup”. This exports to the phones /sdcard/SilenceExport or from mtp “mtp://[usb:xxx,xxx]/Internal storage/SilenceExport”. Copy this directory to the new phone, to the equivalent path (/sdcard/). Install Silence on the new phone. In my experience the first-run import prompt didn’t work so I had to do it manually. Click the three horizontal “…” -> “Import / Export” -> “Restore encrypted backup”. Your done!
I wanted to do this transfer using mtp but neither thunar or nautilus could “see” the SilenceExport directory. I had to do the following:
plug in old phone and copy the export to my laptop: adb pull /sdcard/SilenceExport/
plug in the new phone: adb push SilenceExport/
Remember you have to authorize your laptop from the phone.
THE FOLLOWING INSTRUCTIONS ARE CONSIDERED OUT OF DATE yet still useful for some.
When moving to a new phone you can export a plain text copy of all your sms texts for import on the new phone. This is fine but has many drawbacks. The export feature of smssecure does not export a users encryption keys. Here are instructions on doing this yourself. Its really not as bad as it might seem and is worth doing to avoid re-keying with every friend you text with. You can also avoid exporting sensitive txt’s to plain-text.
PREREQ’s:
Install SMSSecure on the new phone.
You will need adb access to both phones.
Enable developer options by going into Settings -> About phone -> then press “Build number” 7 times.
Allow adb root access from “developer options -> Root access” “Apps and ADB”. Otherwise you will get a permission error like so:
shell@ville:/ $ su -
Permission denied
Also, be sure your computer can see the phone: [jason@local ~] $ adb devices
List of devices attached
393520931D5B00EC device
If usb is broken you can use ADB over the network. Enable it under Developer options -> Android debugging -> ADB over network. Connect to your local wireless first and take out your sim card before enabling. Otherwise your phone will be fully reachable over the cell network/public internet.
Now connect to the phone over the network: [jason@local ~] $ adb connect 192.168.123.12:5555
You can continue as if it was plugged in directly…
To disconnect from the phone use this: [jason@local ~] $ adb disconnect 192.168.123.12:5555
STEPS:
I wasn’t able to directly copy the smssecure directory to my computer. It errors with zero files transferred:
Next, we will copy the org.smssecure.smssecure directory to the new phone:
Note: I attempted to copy directly to the new phone at /data/data but there was a permission denied:
[jason@local ~] $ adb push org.smssecure.smssecure/ /data/data/
push: org.smssecure.smssecure/databases/messages.db-journal -> /data/data/databases/messages.db-journal
failed to copy 'org.smssecure.smssecure/databases/messages.db-journal' to '/data/data/databases/messages.db-journal': Permission denied
Instead copy it to the /sdcard/ of the new phone, first: UPDATE: This doesn’t work anymore
The brilliant developers of adb didn’t write in support for adb push to be recursive? We have to get creative as usual with Android. Use zip if you have unzip on your phone.
[jason@local ~] $ zip -r org.smssecure.smssecure.zip org.smssecure.smssecure/
And then,
adb push org.smssecure.smssecure.zip /sdcard/
adb shell
cd /sdcard/
unzip org.smssecure.smssecure.zip
Before we proceed lets check what permissions the org.smssecure.smssecure directory is currently set to (from the phones shell). In this case its chown u0_a63.u0_a63 and chmod 755:
root@crespo:/ # ls -ld /data/data/org.smssecure.smssecure/
drwxr-xr-x u0_a63 u0_a63 2015-12-06 21:35
Keep note of this for later.
Using the new phone’s root shell you will delete the existing (un-used) directory and copy your smssecure directory (with all your text’s) from sdcard to /data/data:
Note: If you try using move (mv) instead of copy (cp) and your sdcard is on a separate partition you will get this error: root@maguro:/ # mv /sdcard/org.smssecure.smssecure/ /data/data/
failed on '/sdcard/org.smssecure.smssecure/' - Cross-device link
255|
Back to permissions:
After the data is copied to the new phone the directory and file permissions will be wrong (owned by root):
Be sure to reboot your phone.
Note: My SMSSecure disappeared and I needed to re-install. Everything was there after I re-installed and it survived subsequent reboots.
==== key signing ====
You can host the certificate authority on the vyos device itself. This is obviously not as secure as hosting it on a separate system. If someone was to get into the vyos they would have access to all your keys and would be able to sign new keys against the CA. Nonetheless, it is convenient and secure-enough for many sites. This is how it can be done.
From the VyOS, copy the easy-rsa directory into /config, where it won’t get lost on an upgrade.
cp -rv /usr/share/doc/openvpn/examples/easy-rsa/2.0/ /config/easy-rsa2
vi /config/easy-rsa2/vars
edit the options at the bottom of /config/easy-rsa2/vars to personalize them. Increasing KEY_SIZE to 2048 is recommended. Also, you can increase the certificate authority and key expiration. These are the defaults:
...
# Increase this to 2048 if you
# are paranoid. This will slow
# down TLS negotiation performance
# as well as the one-time DH parms
# generation process.
export KEY_SIZE=1024
# In how many days should the root CA key expire?
export CA_EXPIRE=3650
# In how many days should certificates expire?
export KEY_EXPIRE=3650
export KEY_COUNTRY="US"
export KEY_PROVINCE="CA"
export KEY_CITY="SanFrancisco"
export KEY_ORG="Fort-Funston"
export KEY_EMAIL="me@myhost.mydomain"
now load the variables
$ cd /config/easy-rsa2/
$ source ./vars
start fresh in case there is something old lingering around in there. This will delete all keys, etc.
Now you can build the key for the client and distribute to them
use ./build-key to generate a certificate that will connect to the vpn without a pass-phrase and ./build-key-pass if you want the user to enter a pass-phrase before connecting. ./build-key-pass is more secure in case someone steals your certificate and key they will still need to enter a password to connect.
$ ./build-key jimmy
answer all questions accordingly and be sure to answer yes to “Sign the certificate?” the “1 out of 1 certificate requests certified, commit?”
Now copy the keys and certs and create a config for Jimmy to remote in with. This is how I do it.
first make a directory for the client in /config/easy-rsa2/keys
cd /config/easy-rsa2/keys
mkdir jimmy
cp jimmy* jimmy/
cp ca.crt jimmy/
create a client config with your favorite text editor,
vi jimmy/jsvpnserver.ovpn
and add the following
client
proto udp
remote-cert-tls server
verb 2
dev tun0
cert jimmy.crt
key jimmy.key
ca ca.crt
remote [vpn-server host or ip] 1194
From your local computer download the config directory (jimmy) from the vpn server
this copies the jimmy directory into the current directory on your computer (the period designates current directory). If your on MS Windows you will need to use cygwin or putty-scp to do this. Also, if using MS Windows you will need to run unix2dos on those files
from your computer where you downloaded the jimmy folder to.
The server subnet needs to be a different subnet from your LAN. Set it to something unique that will be unlikely on any remote networks your clients will be on.
The “–push route 10.101.101.0 255.255.255.0” needs to be changed to the subnet on the LAN of the router. The one you will VPN in to access.
set interfaces openvpn vtun0 mode server
set interfaces openvpn vtun0 server subnet 10.206.109.0/24
set interfaces openvpn vtun0 server name-server 10.101.101.1
set interfaces openvpn vtun0 server domain-name jasonschaefer.com
set interfaces openvpn vtun0 server push-route 10.101.101.0/24
set service dns forwarding listen-on vtun0
set interfaces openvpn vtun0 tls cert-file /config/auth/js-server.crt
set interfaces openvpn vtun0 tls key-file /config/auth/js-server.key
set interfaces openvpn vtun0 tls ca-cert-file /config/auth/ca.crt
set interfaces openvpn vtun0 tls dh-file /config/auth/dh1024.pem
#Set the firewall to allow openvpn through
set firewall name wan-local rule 40 action accept
set firewall name wan-local rule 40 destination port openvpn
set firewall name wan-local rule 40 protocol udp
commit
save
Thats it! Your done!
STATIC CLIENT ADDRESS
The static ip address for each client is done with this command:
set interfaces openvpn vtun0 server client leroy ip 10.206.109.123
The name “leroy” is the Common Name of the certificate. After this is committed leroy will receive .123 on his laptop each time he connects to the vpn.
ROUTING VARIOUS LAN’s OVER VPN
The routing between a remote LAN (iroute) is done like this:
set interfaces openvpn vtun0 server client leroyhome ip 10.206.109.0.3
set interfaces openvpn vtun0 server client leroyhome subnet 10.101.103.0/24
The “subnet” option is openvpn’s internal route (iroute) function. This tells openvpn what the remote LAN subnet is. In this example, 10.101.103.0/24 is leroys house LAN.
Now you can add a static route in order to route to this remote LAN.
set protocols static route 10.101.103.0/24 next-hop 10.206.109.1
10.206.109.1 is the openvpn routers interface vtun0 ip address.
If your designing a network with multiple remote sites its convenient to make their addresses contiguous and route to them all with a single route.
active site files in /config/auth/
ca.crt,
site2site_active.key,
site2site_active.crt
passive site files in /config/auth/
ca.crt,
site2site_server-passive.key,
site2site_server-passive.crt,
dh1024.pem
==================
#site to site openvpn
set interfaces openvpn vtun9
set interfaces openvpn vtun9 local-address 172.16.9.[1 | 2]
set interfaces openvpn vtun9 remote-address 172.16.9.[1 | 2]
set interfaces openvpn vtun9 remote-host [remote-vpn-host]
#standard openvpn port is 1194, best used for client/server mode. prevents ovpn config from needing port setting
set interfaces openvpn vtun9 local-port 1195
set interfaces openvpn vtun9 remote-port 1195
set interfaces openvpn vtun9 mode site-to-site
set interfaces openvpn vtun9 tls ca-cert-file /root/ca.crt
#the passive site cert/key needs to be signed as a server!
set interfaces openvpn vtun9 tls key-file /root/[passive.key | active.key ]
set interfaces openvpn vtun9 tls cert-file /root/[passive.crt | active.crt]
set interfaces openvpn vtun9 tls role [active | passive]
#dh1024.pem is required on passive host only!
set interfaces openvpn vtun9 tls dh-file /root/dh1024.pem
commit;save
#open firewall for openvpn
set firewall name wan-local rule 20 action accept
set firewall name wan-local rule 20 destination port 1195
set firewall name wan-local rule 20 protocol tcp_udp
#set static routes to networks on other side of vpn, if any
set protocols static route 192.168.7.0/24 next-hop 172.16.9.[1 | 2]
set protocols static route 192.168.8.0/24 next-hop 172.16.9.[1 | 2]
commit;save
DYNAMIC DNS
How to reach your vpn server and avoid using a static ip. We will be using afraid.org to automatically update a hostname’s A record with the public ip of your Vyos VPN server.
You will need to register for an account at https://freedns.afraid.org. Then create a subdomain, for this example it will be myvpnserver.mooo.com. Go to the “Dynamic DNS” section and copy the “Direct URL”.
Create a script under /config/scripts on your Vyos. vi /config/scripts/myvpnserver.mooo.com.sh
#!/bin/bash
/usr/bin/curl -k [paste your direct url here]
Make it executable chmod 755 /config/scripts/myvpnserver.mooo.com.sh
Now tell Vyos to run this with cron. set system task-scheduler task dynamicDNS executable path /config/scripts/myvpnserver.mooo.com.sh
set system task-scheduler task dynamicDNS interval 15m
REVOKE KEYS ON VYOS
vyos@router:~$ cd /config/easy-rsa2/
vyos@router:/config/easy-rsa2$ ./revoke-full jimmy
Using configuration from /config/easy-rsa2/openssl.cnf
Revoking Certificate 0E.
Data Base Updated
Using configuration from /config/easy-rsa2/openssl.cnf
jimmy.crt: /C=US/ST=NM/L=SantaFe/O=Blah/CN=jimmy/emailAddress=jimmy@localhost.local
error 23 at 0 depth lookup:certificate revoked
The error 23 at 0 … is normal and expected.
Now copy the updated CRL (certificate revocation list) to /config/auth vyos@router:/config/easy-rsa2$ cp keys/crl.pem /config/auth/
If this is the first revocation you need to add it to the config as well:
vyos@router:~$ configure vyos@router# set interfaces openvpn vtun0 tls crl-file /config/auth/crl.pem vyos@router# save; commit; exit
Saving configuration to ‘/config/config.boot’…
Done
No configuration changes to commit
exit
VERIFY REVOKED CERTIFICATES
You can verify the revoked keys with the openssl command vyos@router:~$ openssl crl -in /config/auth/crl.pem -text
…
Revoked Certificates:
Serial Number: 08
Revocation Date: Jan 30 00:19:24 2016 GMT
Serial Number: 0E
Revocation Date: Jul 13 22:51:50 2016 GMT
…
In the above example keys 08 and 0E have been revoked vyos@router:~$ cat /config/easy-rsa2/keys/index.txt
…
V 230804225528Z 02 unknown /C=US/ST=NM/L=SantaFe/O=SITC/CN=jason/emailAddress=jason@local
R 230806213443Z 160130001924Z 08 unknown /C=US/ST=NM/L=SantaFe/O=Blah/CN=john/emailAddress=john@localhost
R 260711225142Z 160713225150Z 0E unknown /C=US/ST=NM/L=SantaFe/O=Blah/CN=jimmy/emailAddress=jimmy@localhost.local
…
From this file index.txt, you can ascertain which serial numbers belong to which users!
This outlines a typical VPN implementation with server, clients and routing. Using Attitude Adjustment 12.09. Although, this should work on any new version of OpenWRT. Last tested on version 17.
Start by ssh’ing into the router and installing packages.
Create a “openvpnconfig” directory under config and move the easy-rsa directory into it. The nice thing about this setup is that backups will include all openvpn relevant files automatically, because they are under /etc/config/
root@myWRT:~# mkdir /etc/config/openvpnconfig
root@myWRT:~# mv /etc/easy-rsa/ /etc/config/openvpnconfig/
root@myWRT:~# cd /etc
root@myWRT:~# ln -s config/openvpnconfig/easy-rsa
(this creates a relative symlink)
== CERTIFICATE AUTHORITY FOR OPENVPN ==
edit the following or don’t if you want to enter it manually on certificate creation.
At the end of the /etc/easy-rsa/vars file:
# These are the default values for fields
# which will be placed in the certificate.
# Don't leave any of these fields blank.
export KEY_COUNTRY="US" <-*edit*
export KEY_PROVINCE="CA" <-*edit*
export KEY_CITY="SanFrancisco" <-*edit*
export KEY_ORG="Fort-Funston" <-*edit*
export KEY_EMAIL="me@myhost.mydomain"
export KEY_EMAIL=mail@host.domain
export KEY_CN=changeme <-*edit - servers hostname*
export KEY_NAME=changeme
export KEY_OU=changeme
export PKCS11_MODULE_PATH=changeme
export PKCS11_PIN=1234
root@myWRT:~# clean-all (run this to ensure your starting with a clean slate)
root@myWRT:~# build-ca
root@myWRT:~# build-dh (very slow.. this can be run elsewhere and copied over to the openwrt vpn server **)
root@myWRT:~# build-key-server server (don’t set a challenge password, Answer yes to sign the certificate and yes to commit.)
** building diffie hellman “build-dh” on a fast computer and copying to slow router/computer:
run ps while you are running build-dh to see what command is being used. 12158 root 3304 R openssl dhparam -out /etc/easy-rsa/keys/dh2048.pem 2048
don’t forget to ctrl + c your build-dh command
now go to your fast laptop:
jason@laptop:~$ openssl dhparam -out dh2048.pem 2048
jason@laptop:~$ scp dh2048.pem root@10.11.13.1:/etc/config/openvpnconfig/easy-rsa/keys
Moving on to the config files:
Instead of using UCI syntax we can break this out to be more openvpn standard and troubleshooting friendly. Also, I have added the ability to set static ip’s for the openvpn clients.
== OPENVPN SERVER CONFIG ==
Overwrite /etc/config/openvpn with the following config
The following is the openvpn server config (which is called by /etc/config/openvpn) in /etc/config/openvpnconfig/openvpnWRT.conf
float
port 1194
proto udp
dev tun
comp-lzo yes
cipher AES-256-CBC
tls-version-min 1.2
tls-cipher TLS-DHE-RSA-WITH-AES-256-GCM-SHA384:TLS-DHE-RSA-WITH-AES-256-CBC-SHA256:TLS-DHE-RSA-WITH-AES-128-GCM-SHA256
dh easy-rsa/keys/dh2048.pem
ca easy-rsa/keys/ca.crt
key easy-rsa/keys/server.key
cert easy-rsa/keys/server.crt
#crl-verify /etc/config/openvpnconfig/easy-rsa/keys/crl.pem
ifconfig-pool-persist /tmp/ipp.txt
client-config-dir clients
status /var/log/openvpn-status.log
##Don't enable unless you disable all static ip options below#
#server [10.10.84.0 255.255.255.0]
##begin VPN options for static ip mode (mode server)##
mode server
tls-server
topology subnet
push "topology subnet"
ifconfig 10.11.14.1 255.255.255.0
route-gateway 10.11.14.1
push "route-gateway 10.11.14.1"
ifconfig-pool 10.11.14.10 10.11.14.254 255.255.255.0
##end VPN options for static ip##
##general LAN options##
push "route 10.11.13.0 255.255.255.0"
push "dhcp-option DOMAIN lan.local"
push "dhcp-option DNS 10.11.13.1"
client-to-client
mute 5
log /tmp/openvpn.log
verb 5
keepalive 10 120
persist-key
persist-tun
If you want to enable the static client ip assignments, be sure to uncomment the client-config-dir above and make a directory as such.
root@myWRT:~# mkdir /etc/config/openvpnconfig/clients
write a file inside the clients directory with the same name as the “common name” of the openvpn client certificate.
For example, /etc/config/openvpnconfig/clients/jason ifconfig-push 10.11.13.123 255.255.255.0
This will reserve 10.11.13.123 for user jason.
Enable and start the service:
root@myWRT:~# /etc/init.d/openvpn enable
root@myWRT:~# /etc/init.d/openvpn restart
check for errors
root@myWRT:~# cat /tmp/openvpn.log
Sometimes its helpful to run it by hand, for troubleshooting.
root@myWRT:~# cd /etc/config/openvpnconfig/
root@myWRT:~# openvpn openvpnWRT.conf
== NETWORKING SETUP ==
Now lets setup the tun interface so that we can add zones
in /etc/config/network or in LuCI.
config interface 'vpn0'
option proto 'none'
option ifname 'tun0'
== FIREWALL ZONE ==
Create a zone called openvpn_zone with vpn0 network.
in /etc/config/firewall
config zone
option input 'ACCEPT'
option output 'ACCEPT'
option name 'openvpn_zone'
option network 'vpn0'
option forward 'REJECT'
We now explicitly declare the forwards like this.
openvpn_zone to lan zone allow
config forwarding
option dest 'lan'
option src 'openvpn_zone'
openvpn_zone to wan allow, if you want openvpn clients to use the wan for example if using redirect-gateway
config forwarding
option dest 'wan'
option src 'openvpn_zone'
openvpn_zone to lan allow
config forwarding
option dest 'openvpn_zone'
option src 'lan'
== FIREWALL TRAFFIC RULE ==
Allow the openvpn server to accept connections from clients out in the world.
config rule
option target 'ACCEPT'
option src 'wan'
option proto 'udp'
option dest_port '1194'
option name 'openvpn2device'
option enabled '0'
An overview of traffic rules
** NOTE: Occasionally, I have had to reboot for the above zone’s to work **
== Optional firewall rules to use, instead of using the zones. Not recommended ==
in /etc/firewall.user
# This file is interpreted as shell script.
# Put your custom iptables rules here, they will
# be executed with each firewall (re-)start.
# Allow all traffic in and out of the tun interface.
iptables -A input_rule -i tun+ -j ACCEPT
iptables -A output_rule -o tun+ -j ACCEPT
# This rule will allow traffic towards internet from tun
iptables -A forwarding_rule -i tun+ -j ACCEPT
iptables -A forwarding_rule -o tun+ -j ACCEPT
== USER CONFIGURATION ==
root@myWRT:~# build-key jason
or you can run build-key-pass to issue a key that asks the user to enter the password before it is used (more secure).
Once you have completed the build-key, being sure to answer yes to signing the certificate and commit.
Now you need to get the keys for jason and the ca.crt (not ca.key!). Each client needs these files to connect.
You can secure copy them to your computer similar to this:
root@myWRT:~# scp /etc/config/openvpnconfig/easy-rsa/keys/jason.* you@[your computer]:jason-vpn
root@myWRT:~# scp /etc/easy-rsa/keys/ca.crt you@[your computer]:jason-vpn
Then create the client config in the same directory as the crt’s and keys. Call it jason.ovpn and place it in our jason-vpn directory. This is what you will populate jason.ovpn with:
nobind
float
comp-lzo
cipher AES-256-CBC
dev tun
remote [your-server-name] 1194 udp
client
tls-exit
ca ca.crt
cert jason.crt
key jason.key
remote-cert-tls server
mute 5
resolv-retry infinite
#explicit-exit-notify
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
#redirect-gateway def1
Test the server by connecting from off-site.
cd into your local config directory where your certs, keys and .ovpn config are. sudo openvpn jason.ovpn
enter your sudo password
You should see something like this at the end of the openvpn output:
Fri Feb 28 22:19:01 2014 /sbin/ifconfig tun0 10.11.14.4 netmask 255.255.255.0 mtu 1500 broadcast 10.11.14.255
Fri Feb 28 22:19:01 2014 /sbin/route add -net 10.11.13.0 netmask 255.255.255.0 gw 10.11.13.1
Fri Feb 28 22:19:01 2014 Initialization Sequence Completed
and you will see a newly built tun interface
# ip a
38: tun0: mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 100
link/none
inet 10.11.14.4/24 brd 10.11.14.255 scope global tun0
valid_lft forever preferred_lft forever
and the correct route has been pushed to you
root@myWRT:~# ip r
10.11.13.0/24 via 10.11.13.1 dev tun0
== REVOKING A USERS KEY ==
uncomment the following line in /etc/config/openvpnconfig/openvpnWRT.conf: crl-verify /etc/config/openvpnconfig/easy-rsa/keys/crl.pem
run “revoke-full” with users key as argument:
revoke-full [key-to-revoke]
or run my script easyrsa-user-setup-openwrt.sh [username] revoke
then restart the vpn service so changed take effect:
root@myWRT:~# /etc/init.d/openvpn restart
Let me know in the comments or use the contact form if you find errors or need clarification. This is a concise tutorial and might need some clarification.
I have a Dreamhost VPS account and have been running out of memory and experiencing the dreaded forced reboots dh impose. I found it difficult to identify the offending sites that take up all that memory on my server. Every time I login and run top it was too late or I would find a website being crawled by a search bot. How to find a trend over time, without getting too complicated? My solution was to track the memory usage with ps and write that to individual files, then sort all those files and derive the top offenders in one list. Which is web accessible (or not) for easy viewing later. If my VPS reboots, I can go back to the individual files before the forced reboot and get details of whats causing the problem.
#!/bin/bash
#no trailing slash. Be sure this dir exists.
path=/home/jason/jasonschaefer.com/memusages
logfile=index.txt
#how many days to keep files, remove after..
removeafter=5
# ps -[e]everything, [o]format
# rss is resident set size in kilobytes
# user:20 username with 20 char space so it won't revert to uid on usernames longer than 8 chars
# cmd:40 running command with 40 char column, stime=start time of cmd
# [h]hide headers, --sort=rss sorts on rss column
/bin/ps -eo rss,user:20,cmd:40,stime,pid h --sort=rss > $path/mem`date +"%F_%k-%M"`.txt
# sort unique, numeric on column 5 the pid, so we don't show duplicate processes.
# then sort numeric, reverse on column 1 the memory usage, write the top 200 lines to our logfile.
/usr/bin/sort -u -n -k 5 $path/mem*.txt | /usr/bin/sort -n -r -k 1 | head -n 200 > $path/$logfile
#find files older than $removeafter days and remove them
/usr/bin/find $path -mtime $removeafter -exec rm -fr {} \;
Don’t forget to make it executable
chmod 755 /home/jason/memusages.sh
Then setup to run in cron, to run every ten minutes of every hour, every day, every month and every day of week. Change as needed.. I have it running every minute right now. Depending on your setup you may need to run this as root to see all system processes.
crontab -e
*/10 * * * * /home/jason/memusages.sh
[] Vim reference :e filename (open filename)
:q! (quit, don’t save)
:w (write/save)
:wq (write and quit)
:x (write if changed, otherwise exit)
:changes (show list of edits in the buffer)
a (insert after)
A (insert after line)
h j k l (left, down, up, right)
$ (move to end of line)
^ or 0 (move to beginning of line)
G (move to end of file)
gg (move to top of file)
gUU (uppercase entire line)
guu (lowercase entire line)
gUw (uppercase a word, u lowers)
:n (move to “n” line, n=number)
x (delete to the right)
X (delete to the left)
D (delete to the end of line)
dd (delete current line)
yy (yank/copy current line)
V (begin highlight, up and down to select “y” to yank selection)
vn (yank “n” lines below cursor, n=number)
p (put/paste)
u (undo)
ctrl+r (redo)
/string (search for “string”)
n (search for next string match) :s/yellow/green/gc (replace yellow with green on current line, g is for global, each match is replaced in a line, instead of the first match in a line. c is for confirm/ask) :%s/yellow/green/g (replaces yellow with green on the entire page) :%s:/usr/local/bin:/opt/users/bin:g (use something other than / as delineation so you don’t have to escape “/”. Like this nasty example: :s/\/usr\/local\/bin/\/usr\/loca\/bin) :%s#http://jasonschaefer.com#https://jasonschaefer.com#g (Switch the delimiter to # for strings with : and / to avoid annoying escapes!) :2,$s/ */ /g (After the 2nd line, replace 1 (2 spaces with a *) or more spaces with 4 spaces, globally)
strftime $ echo the day and time is currently `date +"%a at %T"` https://foragoodstrftime.com/
Sat, Jun 23 22:01:40 “%a, %b %d %T”
Sat, Jun 23 2018 7:04 PM “%a, %b %d %Y %l:%M %p”
[] bash tricks
stop bash history: unset HISTFILE
[] find command find . -name "name" -exec [command goes here] {} \; find . -type d -exec chmod 750 {} \;
recursively changes type directories to user=rwx, group=r-x, other=— find . -type f -exec chmod 664 {} \;
recursively changes type file user=rw-, group=rw-, other=r– (so that files are not executable) find /home/BACKUP -mtime +14 -exec rm -fr {} \;
-mtime options:
n exactly n days
+n more than n days
-n less than n days
find files that are newer than specified date time:
find /path/ -newermt 2018-01-15
use -ls to output long listing of matches
find /path/ -newermt “may 21 2018 16:00” -ls
yesterday or today can be used instead
find files between a date reference
finds files between 16:00 and 16:47 on may 21
find /path/ -newermt “may 21 2018 16:00” ! -newermt “may 21 2018 16:47”
to convert all backslash \ to forward slash / find . -type f -iname *.xml -exec sed -i 's:\\:/:g' {} \;
find hard links (directories have multiple links so use type file and not with 1 link)
find /path -type f ! -links 1
find directories, with emails, with less than 330 files (emails)
find /home/user/Maildir/ -type d -name cur -exec tree -RaFC –filelimit 330 {} \;
find quantity (310-320) of files under any directory named cur
find /home/user/Maildir -type d -name cur -exec bash -c “echo -n ‘{} ‘; ( ls ‘{}’/ | wc -l )” \; | grep ‘ 3[12][0-0]$’
find -exec has two variants:
this variant runs the command (echo in this case) once per match -exec echo '{}' \;
echo ./match1
echo ./match2
echo ./1match
and this runs the command once against all matches -exec echo '{}' \+
echo ./match1 ./match2 ./1match
[] Image Conversions and Resizing in batch groups
This will resize all jpg’s in the current directory “.” to 1024×768 and put them in the directory small find . -iname "*.jpg" | xargs -l -i convert -resize 1024x768 {} small/{} or a better use of find and convert would be
find -iname "*.jpg" -exec convert -resize 20% {} {}_small.jpg \;
You can replace the “convert -resize” with convert -quality 85% to compress the images instead.
merge multiple images into one pdf convert blah-page1.png blah-page2.png blah.pdf
wildcards work convert *.png blah.pdf
convert is a part of imagemagick
[] Edit EXIF data on images
Using exiftool to shift wrong date caused by a camera with the wrong time. man Image::ExifTool::Shift.pl for a manual.
example: exiftool "-AllDates+=1:0:21 0:0:0" *.JPG
This adds (+=) 1 year, 0 months and 21 days, 0 hrs, 0 min, 0 sec to all files ending in .JPG
[] tar
tar with various exclude examples tar zcfv backup-website.tar.gz --exclude=stuff --exclude=path/to/stuff --exclude="more stuff with spaces in the name" --exclude=*.wild /home/website
[] chmod tricks
chmod can be used in a way where it preserves executable permissions, if they are already present. using upper X chmod -R u=rwX,g=rwX,o=rX /path/to/
This recursively (-R) sets user and group to rw- dirs and files that don’t already have executable permissions (the X is similar to x but preserves executable perms if they were preexisting). If the dir or file has any executable permissions then it sets user and group to rwx. This can be handy if you want to change lots of files and dirs at once but not make files executable. For instance, chmod -R u=rwx,g=rwx would blanket files and dirs making them all executable. Of course, using find with type and exec can more explicitly set permissions. But it will reset any executable files as well. Thus the benefit of chmod and X.
To see what files are executable, if any, do find /path/ -executable -type f
Proper permissions for wordpress: chown www-data:wwwmaster -R /home/www/ find /home/www/ -type f -exec chmod g=rws {} \; find /home/www/ -type d -exec chmod g=rwxs {} \;
[] How to recursively force a group permission + umask per user on gnu/linux.
First you can recursively set the desired owner and group. Recursive is optional, only needed if you have sub dirs. chown -R root.users /path/to/dir
Then force all files and directories created under /path/to/dir to be owned by the creator and the group will be set to “users” group. Notice the chmod g+rwxs is adding the (s)etGID bit for the group. find /path/to/dir -type d -exec chmod g+rwxs {} \;
You will notice that when a user now creates files or directories under /path/to/dir they come out as (on a typical system) -rw-r--r-- 1 jason users 0 Aug 29 16:49 this is a file
drwxr-sr-x 2 jason users 6 Aug 29 16:49 this is a directory
You will need to change your umask. You can find it under your home directory in ~/.profile
Uncomment or add umask 002 so that the group “users” will be able to read your files and execute your directories.
[] changing default new file or directory permissions, umask on debian wordaround for http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=646692
change /etc/login.defs UMASK 022 (equivalent to 644/rw-r–r– files and 755/rwxr-xr-x directories)
to UMASK 002 (equivalent to 664/rw-rw–r– files and 775/rwxrwxr-x directories)
If that doesn’t work. Enable the pam_umask module like this. echo "session optional pam_umask.so usergroups" >> /etc/pam.d/common-session
[] Cleanup /etc/passwd and /etc/group
sort /etc/passwd in place pwck -s
sort /etc/group in place grpck -s
[] Rename multiple files
remove space from all files ending in .mp3 rename 's/ //'g *.mp3
rename all files ending in .ZIP to .zip
rename 's:\.ZIP:\.zip:' *.ZIP
[] search and replace text within a group of files
find /etc/NetworkManager/system-connections/ -exec sed -i "s/mac-address=0:1e:4c:27:40:00/mac-address=EC:55:F9:0F:5D:00/g" {} \; use -name, -type to refine the match if you need.
jedit is a gui program that can do this.
[] migrating Network Manager system-connections to new computer
You can replace the interface-name and the mac-address as shown above or remove them entirely.
sed -i ‘/^interface-name/d’ /etc/NetworkManager/system-connections/*
sed -i ‘/^mac-address/d’ /etc/NetworkManager/system-connections/*
On the new machine “systemctl stop NetworkManager” and “rsync -a” them to the new machine. Then start network manager “systemctl start NetworkManager”
[] SSH stuffs
copy your ssh public key to remote hosts ~/user/.ssh/authorized_keys file ssh-copy-id user@host
Use ‘ssh-add’ to add your private key to the ssh-agent, so you don’t need to type your passphrase each time you ssh someplace.
create a listen socket on your local computer, that redirects port to a host on remote network (10.9.8.2) ssh -L localhost:3389:10.9.8.2:3389 user@host -L [bind_address:]port:host:hostport
(now you can rdesktop to localhost and it will connect to the remote 10.9.8.2) rdesktop -g90% localhost
create a remote socket that forwards port 2222 on localhost of server to port 22 on initiating host.
Useful for remote support sessions. ssh -R localhost:2222:localhost:22 user@hostname -R [bind_address:]port:host:hostport
(now you can ‘ssh -p2222 localhost’ from the server and reach the host)
create a socks4/5 proxy over ssh ssh -D8080 user@hostname
(now you can configure your browser to use socks5 proxy at localhost:8080 and you can reach the remote networks web servers, or just use it to securely proxy your web traffic through the remote hosts internet connection)
[] Sending mail with telnet: telnet hostname 25
helo me
mail from:myaddress@mydom.com
rcpt to:youraddress@yourdom.com
data
This is a test
.
(thats a newline [enter] – period – and another newline [enter])
echo Cleanup .bak files older than 7 days forfiles /p d:\backup /m *.bak /d -7 /c "cmd /c del /q @path"
echo Set variable date as yyyymmdd set date=%date:~-4,4%%date:~4,2%%date:~-7,2%
echo %date%
[] Filesystem stuff
make clone image of sda dd if=/dev/sda of=/dev/sdb bs=4096 conv=notrunc,noerror
notrunc or ‘do not truncate’ maintains data integrity by instructing dd not to truncate any data.
noerror instructs dd to continue operation, ignoring all input errors. Default behavior for dd is to halt at any error. Useful when imaging damaged drives.
bs=4096 sets the block size to 4k, an optimal size for hard disk read/write efficiency and therefore, cloning speed.
check status of dd transfer (use pgrep to find process id, kill to send user define signal 1, dd progress will be displayed on terminal where dd was run) or pass “status=progress” to dd command
kill -USR1 $(pgrep ^dd)
mount image losetup /dev/loop0 sda.img
mount /dev/loop0 /mnt
Initialize all SATA buses: for x in /sys/class/scsi_host/host*; do echo "- - -" > $x/scan; done
xfs filesystem and xfsprogs
Determine the amount of fragmentation on sda2 xfs_db -c frag -r /dev/sda2
Filesystem re-organizer, by default, with no arguments. It re-organizes files in mounted partitions for 2 hours. Use -t to change the time.) xfs_fsr
These tools reside in the xfsdump package
[] Recover Files
testdisk (recover lost partitions)
photorec (part of the testdisk suite)
foremost sda.img
-t (type doc,jpg,exe etc. all is default)
-a (no error detection, recovers partial files)
-d (indirect block, use for nix filesystems)
-o (output dir)
-T (timestamp output dir)
Interface config for bridging to virtualized client /etc/network/interfaces
wireless interfaces rarely ever support bridging.
# The primary network interface
iface eth0 inet manual
auto br0
iface br0 inet dhcp
bridge_ports eth0
bridge_stp off
bridge_waitport 0
bridge_fd 0
Resize/Add storage to kvm image: dd if=/dev/zero of=myvirtualhost.img bs=1M count=78k oflag=append conv=notrunc
notrunc MUST be used or else append will overwrite beginning of image.
Alternately, creates a sparse file which suffers from fragmentation and possible corruption if host system doesn’t provide proper space for the sparse image to fill into. So its not recommended. truncate -s +10G image.raw
Alternate method sparse: qemu-img create -f raw addon.img 10G
be sure to make a backup of original.img
now you can append addon.img to original.img cat addon.img >> original.img
Now, boot the .img vm and use cfdisk to partition the new space. Reboot, and build a filesystem OR boot the instance with a live distro that has gparted and merge/resize the new partition with the old.
Convert a qcow2 image to raw image and remove the sparsity (-S 0 is non-sparse). qemu-img convert -p -O raw -S 0 win7pro.qcow2 win7pro.img
Also, fallocate is a great way to allocate kvm images. “preallocation is done quickly by allocating blocks and marking them as uninitialized, requiring no IO to the data blocks. This is much faster than creating a file by filling it with zeroes” fallocate -l 128GiB virtualhost.img
write zeros to the image so that you can make a sparse copy of the image for backup. Without the zeros the sparse copy won’t have the ability to sparsify dd if=/dev/zero of=virtualhost.img conv=notrunc,fdatasync bs=1M count=128K
preferred method for expanding a disk image (make a backup!!) original.img is 64GB and you want to expand it to 128 fallocate -l 128GiB original.img use gparted to expand the partition and filesystem into the extra space.
Backup virtual images into a sparse file to save space cp --sparse=always winblows7.img SNAPSHOTS/winblows7.img_oct06-2017
Be sure to unsparsify the file if you need to restore it! cp --sparse=never SNAPSHOTS/winblows7.img_oct06-2017 winblows7.img
[] KVM with Windows
The best way to get virtio is on install. Download the block driver floppy image and attach it, I use virt-manager. Set your hard drive to type virtio and start your windows install. It will will prompt you to press f6 to install third party drivers. Then press S (you have a disk from a third party manufacturer, your floppy image)
If you already have a disk type IDE and want it to be virtio (better). Then do this:
1. Create a temporary image kvm-img create -f qcow2 temp-virtio.img 1G
2. Shutdown your virtual machine and attach temp-virtio.img as a hard drive, as type virtio.
3. Attach the virtio-win-x.x.x.vfd (i used the one from fedoraproject.org, see below) to you virtual machine
4. Boot up and install the drivers
5. Shutdown, remove the old hard drive image and re-add it as type virtio
6. Boot up and since you already installed the drivers it will boot. Otherwise, you get BSOD..
(You can remove the temp-virtio.img and floppy image).. All done.
For network drivers.. Shutdown, set the “device model” to virtio. Attach the NETKVM-xxxx.iso as a cdrom. Bootup and install drivers. yay!
When using raw disk images, the “live” backup options are almost non-existent. The pros of raw img performance may outweight the cons of backup. Here is a simple way to backup .img virt images with minimal downtime:
rsync -a --inplace --progress --stats [source.img] [destination.img]
md5sum [source.img]
virsh start [source.img]
md5sum [destination.img]
[] Debian package tricks
list all packages that come from a particular repository, this case “testing”
for p in $(dpkg -l | awk ‘/ii/{ print $2 }’); do for i in $(apt-cache policy “$p” | awk ‘/Installed/{ print $2}’); do apt-cache policy “$p” | grep -A1 ‘\*\*\*\ ‘$i” | if grep -q testing; then echo $p; fi; done; done
[] Windows Policy
run gpedit.msc to edit policy
to backup or move to new host, copy the following
%systemroot%\system32\GroupPolicy\Machine and User dirs
to apply changed policy’s gpupdate /force
[] RDP tricks
plain old vnc is no more, not to say its not useful but xrdp is a super combo rambo pack. when it comes to ease of use, autostart scripts in debian, built in encryption, performance and cross platform the xrdp project rules the roost. Follow these simple steps.
on server: apt-get install xrdp
done;
on client: rdesktop -g95% [server name or ip]
-g is for geometry, look it up in man rdesktop
done; wow!
obviously, you will need rdesktop or some other remote desktop protocol installed on the client.
If you have issues with the arrow up and down keys minimizing and maximizing your X terminal do the following:
In gnome, use gnome-control-center -> Go to keyboard ‘Shortcuts’ tab, ‘Windows’ on the left pane -> select super+up and super+down shortcuts -> press backspace to disable these shortcuts on these actions.
xfreerdp is a fork of rdesktop that has newer features like certificate verification that comes with new versions of MS Windows.
SeamlessRDP http://www.cendio.com/seamlessrdp/ rdesktop -A -s "c:\seamlessrdp\seamlessrdpshell.exe c:\program files\internet explorer\iexplore.exe" -u username -p password hostname
uhhh, for the record I have NEVER gotten this to work properly. Please contact me if you have!
[] Self Signed certificate on debian, the easiest way possible make-ssl-cert /usr/share/ssl-cert/ssleay.cnf /etc/ssl/private/hostcert.crt
This script will ask for a domain and write the certificate. to /etc/ssl/private/hostcert.crt
flashing with atftp (follow the instuctions for particular device at http://wiki.openwrt.org/toh/start) curl -T openwrt-xxxx-xxxxx-squashfs-factory-xxxx.img tftp://192.168.1.1
or the more complicated annoying way atftp --trace --tftp-timeout=1 --put --local-file openwrt-xxxxx-xxxxx.img 192.168.1.1
Setup SSL/TLS (https) for Luci web interface and disable insecure plaintext (http) opkg install luci-ssl
be sure the following is commented out in /etc/config/uhttpd # HTTP listen addresses, multiple allowed
# list listen_http 0.0.0.0:80
# list listen_http [::]:80
Also change the cert px5g options to be more unique and add more days to the self signed certificate. /etc/init.d/uhttpd restart
Disable/Enable Wireless on a schedule, automatically
The first line will use bridge control to remove the wireless interface (wlan0-1) from the lan bridge (br-lan) at 22:30. The next cron will add the interface back at 6:00. Redirect (>) all output to dev null. Substitute wlan0-1 for whichever interface you need to. Add this to crontab: 30 22 * * * brctl delif br-lan wlan0-1 >/dev/null 2>&1
0 6 * * * brctl addif br-lan wlan0-1 >/dev/null 2>&1
use “brctl show” to see which interfaces are in the bridge:
root@OpenWRT:~# brctl show
bridge name bridge id STP enabled interfaces
br-lan 7fff.c6031578e51d no eth0.1
wlan0
wlan0-1
[] Cron and wget with Afraid free DNS
Its best to use curl. Install curl with opkg update and then opkg install curl or apt or yum, etc. */10 * * * * /usr/bin/curl -k https://freedns.afraid.org/dynamic/update.php?[random string]
wget is overly complicated… but if its all you got, then its great. */10 * * * * /usr/bin/wget --no-check-certificate -O - https://freedns.afraid.org/dynamic/update.php?[random string] > /dev/null 2>&1
add to /etc/rc.local so that it updates immediately on bootup. This doesn’t always work if the wan interface isn’t operational at time of execution.
# Put your custom commands here that should be executed once
# the system init finished. By default this file does nothing.
/usr/bin/curl -k https://freedns.afraid.org/dynamic/update.php?[random string]
exit 0
[] MYSQL
mysql> create database newdb;
mysql> CREATE USER 'newuser'@'localhost' IDENTIFIED BY 'password';
mysql> GRANT ALL PRIVILEGES ON newdb . * TO 'newuser'@'localhost';
(GRANT ALL PRIVILEGES ON [database name].[table name] TO ‘[username]’@’localhost’;)
mysql> SET PASSWORD FOR 'newuser'@'localhost' = PASSWORD('newpassword');
mysql> DROP USER 'newuser'@'localhost';
update table g2_PluginMap and set column g_active to 0 where column g_pluginID is captcha. This disables the captcha plugin in gallery2 UPDATE g2_PluginMap SET g_active = '0' WHERE g_pluginId = captcha; DELETE FROM g2_FactoryMap WHERE g_implModuleId='captcha';
(unrelated to mysql, you will need to clear the cache to fully disabled this plugin -> http://your-domain.tld/gallery/lib/support/index.php
[] RSYNC
rsync highlights:
typical use — rsync -av --delete --stats --exclude media/* /home/ /mnt/usb/rsync-home-mirror
copy [a]rchive, [v]verbosely, –delete any files on the destination that don’t exist in source (mirror), show transfer [stat]istics, [exclude] any files inside directory media, copy contents of /home/ into /mnt/rsync-home-mirror
advanced use — useful for copying entire OS rsync -aAHXi --super --numeric-ids /source/ /destination
-a archive, -A copy ACL, -H copy hard links, -X copy extended attributes, -i show changes, –numeric-ids preserves uid and gid numerically instead of by name.
Use -n to do a dry run! Especially valuable when using –delete switch.
You can think of a trailing / on a source as meaning “copy the contents of this directory” as opposed to “copy the directory by name”
[] fstrim
fstrim one-liner for cron. every sunday at 12:30 timestamp the log and include two partitions, / and /home. 30 12 * * 0 /bin/date +\%c > /tmp/fstrim.log && /sbin/fstrim -v / >> /tmp/fstrim.log 2>&1 && /sbin/fstrim -v /home >> /tmp/fstrim.log 2>&1
[] wget
for range in {1..7};do wget http://URL/Episode$range.mp3 ; done for range in {{1..3},{5..7}};do wget http://URL/Episode$range.mp3 ; done
Mirror entire site with wget wget --mirror -p --convert-links http://URL
or wget --recursive --no-clobber --page-requisites --html-extension --convert-links --no-parent --domains website.org http://website.org [] ffmpeg
convert video to webm ffmpeg -i be-hose.mp4 -acodec libvorbis -aq 5 -ac 2 -qmax 25 -threads 2 be-hose.webm
[] booting os manually from grub2
set root=(hd0,gpt2)
linux /boot/vmlinuz-2.6.18-6-686 root=/dev/sda1
initrd /boot/initrd.img-2.6.18-6-686
boot
[] booting into password recovery using grub
new methond: append systemd.debug_shell to linux line of grub. system boots normal and ctrl+alt 9 will have root shell that bypasses password prompt.
old method: append init=/bin/bash to linux line of grub. system boots in read only mode mount -o remount,rw / and run passwd to change password.
[] Text Manipulation, simple and common techniques
file.txt contains:
a b c d e
f g h i j
awk '{print $2 "\t" $5}' file.txt
b e
g j
awk '{print $2","$5}' file.txt
b,e
g,j
cut -d " " -f 2 file.txt
b
g
cut -d " " -f 2,4 file.txt
b d
g i
cut -d " " -f 3-5 file.txt
c d e
h i j
MATH with awk!
file.txt contains:
1 2 3 4
5 6 7 8
awk '{print $2 + $4}' file.txt
6
14
awk '{print $2 * $3}' file.txt
6
42
[] EXIM
consult the oracle: zless /usr/share/doc/exim4-base/README.Debian.gz
dpkg-reconfigure exim4-config
update-exim4.conf; service exim4 stop; service exim4 start
Enable tls:
enable the following in /etc/default/exim4 SMTPLISTENEROPTIONS='-oX 25:465:587 -oP /run/exim4/exim.pid'
enable plain_server: PLAIN and LOGIN in /etc/exim4/exim4.conf.template
swaks -a -tls -q HELO -s smtp.schaeferconsulting.com -au test -ap '<>'
for x in $(mailq | grep frozen | sed -e ‘s/^……….//’ -e ‘s/ .*//’); do exim -Mrm $x; done
[] GPG
jason gets the public key for geoff from keyserver gpg --keyserver keys.openpgp.org --recv-keys 6058D99C
or gpg --keyserver keys.openpgp.org --search-keys email@address
signs the key with his B0EE80C1 key gpg --default-key B0EE80C1 --sign-key 6058D99C
export the signed key to send to geoff gpg --output 6058D99C.asc --export --armor 6058D99C
send geoff_6058D99C.asc to geoff
———-
geoff receives key and imports gpg --import geoff_6058D99C.asc
sends the updated key to keyserver gpg --keyserver keys.openpgp.org --send-keys 6058D99C
These are the manual instructions for installing and configuring OpenVPN client.
After having many problems over the years with openvpn gui breaking, I have devised a reliable work around. Create the following batch script, name it something like ovpnConnect.bat, and right click to “Run as Administrator”. This does require the openvpn gui from openvpn.net to be installed. It calls the openvpn.exe. You can also add multiple vpn client connections!
@echo off
net session >nul 2>&1
if %errorLevel% == 0 (
echo Success: Administrative permissions confirmed.
) else (
echo Failure: Please right click and "Run as administrator"
pause
exit 0
)
cd "c:\Program Files\OpenVPN\config\" && "c:\Program Files\OpenVPN\bin\openvpn.exe" [userconfig].ovpn
Go to Compatibility tab and select “run this program as administrator”
(It requires admin priviledges because it needs to write routes for the new VPN tunnel. Without, it will connect but won’t know how to reach the other network.)
Get your OpenVPN configuration files from the system administrator. Copy and paste them into C:\Program Files\OpenVPN\config
ca.crt (certificate authority)
jason.crt (client certificate)
jason.key (client secret key)
jason.ovpn (client configuration). Here are the contents of jason.ovpn:
client
dev tun0
cert jason.crt
key jason.key
ca ca.crt
remote vpn.server 1194
Now you can launch OpenVPN GUI by double clicking the icon on the desktop (the one we just set to run as admin). This will launch the OpenVPN GUI into the system tray.
Click on system tray, right click on OpenVPN GUI, left click “Connect”.
If all goes well, you will successfully connect. The OpenVPN GUI systray icon turns green. Now you are on the Virtual Private Network!
The wonderful world of motherboard BIOS updates, is still old fashioned. Updates are often still built for Microsoft Windows environments. Those of us who don’t have MS Windows, DOS, a floppy drive, an install of Windows 98 to create a bootable floppy, or cheesy Pâté for that matter ….. Here is a way one can flash that BIOS of your mobo using, our favorite free software licensed, operating systems and tools.
DISCLAIMER: Don’t attempt this unless you know what you are doing. I have never had problems doing this, BUT many things can go wrong and you CAN easily “brick” your hardware. Proceed at your own risk!!
We will be using FreeDOS, a wonderfully free and royalty exempt Microsoft DOS compatible operating system. Licensed under the General Public License (GPL).
Note: As usual, my posts require some knowledge of the command line.
====================== UPDATE: Because the 1.4mb and 2.8mb FreeDos disk images are not large enough for most modern BIOS flashing utilities and payloads you will need to build a custom disk image of freeDos.
The easiest way to do this is to write freeDos onto a usb flash drive. Here is an example of how to do this.
write the FD12LITE.img to a USB stick. I used gnome-disks (right click and open with “Disk Image Writer” or you can use dd.
mount the usb stick and copy the bios update onto it. If your bios update is too large to fit you will need to resize.
run “sudo gparted” resize the fat16 partition. I chose 100mb. Apply
now copy the bios update and boot from the usb. Don’t run the DOS installer, instead run DOS. Now you can execute the bios update. Done!
Another option would be to install FreeDOS using qemu. This is more complicated but I’ll leave the instructions here for reference.
Download FreeDOS Standard CDROM Installer at http://www.freedos.org/download/
sudo qemu-system-x86_64 -cdrom FD12CD.iso /dev/sdb -boot d
Go through the DOS installer. You can use it to partition the usb stick. I made a 700MB fat 16 partition and marked it “active”. I said no to the fat32 LBA because I had issues, the first time I tried this, reading files that I had put on the fs while booted into DOS. Fat32 should be ok, try it and let me know your experience. After this it will ask to restart the install. Install to hard disk again and now you partition the c: drive. Then install the full version. Done.
If you need more space you can always run cfdisk, gparted or fdisk to add a large fat32 partition on the usb stick. This would show up as d:\ while booted up in FreeDOS. You can copy your BIOS files to the usb DOS partition and boot into FreeDos to run them. Yay! No more floppy space issues!
======================
And feel free to try the old floppy way. I’ll leave it up here for reference.
wget http://www.fdos.org/bootdisks/autogen/FDOEM.144.gz
gunzip FDOEM.144.gz
mkdir floppy
sudo mount -o loop FDOEM.144 floppy/
ls floppy
you should see these files:
AUTOEXEC.BAT COMMAND.COM CONFIG.SYS KERNEL.SYS README sys.com
Download your BIOS update file from the manufacturer or vendor.
Note: Sometimes, the update will be distributed as a .exe (Windows Executable) file. Most likely it will actually be a compressed zip archive. You can use unzip to extract the .exe file.
Download the update using wget, then unzip the resulting image file “FDOEM.144” into the mounted folder: “floppy/”
In this case, the following files are extracted from the BIOS_UPDATE.zip file:
inflating: BIOS.WPH
inflating: OEMPHL.EXE
inflating: OPTIONS.BAT
inflating: PHLASH16.EXE
inflating: releasenotes.txt
inflating: 1.BAT
now, move to the previous directory (cd ..), and un-mount the FDOEM.144 image:
Recently, I was installing debian on a new server and grub2 would not install gave me this error:
“This GPT partition label has no BIOS Boot Partition; embedding won’t be possible! grub-setup: error: Embedding is not possible, but this is required when the root device is on a RAID array or LVM volume.”
Of course, it being grub2, I jumped to the conclusion that grub2 was the problem. I installed legacy grub and got nowhere.
Doing the usual
grub> root (hd0,0)
grub> setup (hd0)
produces this error:
“file /boot/grub/stage1 not read correctly”
No matter what I did, it would not install. So, I went back to the original message and gave grub2 its due process. Turns out this Dell T410 uses GPT (GUID Partition Table) which is an extension of EFI. The “BIOS Boot Partition” is an actual partition on the hard drive. Grub2 embeds the core.img (multiboot boot kernel) into this BIOS boot partition instead of the MBR.
Here are two great resources on this subject: http://www.rodsbooks.com/gdisk/index.html and http://grub.enbug.org/BIOS_Boot_Partition
So, the solution:
I had to re-install Debian with a small partition. Apparently it can be under a few hundred KiB. Space is cheap and I didn’t want to have more problems, so I made mine 5MB and put it at the beginning of the disk. In the Debian partitioner, set the partition under “use as:” to “Reserved BIOS boot area“. Then continue with the rest of your partitions and install. Grub2 installed with no problems this time!
If using an older version of Debian, lenny (v5) or older. The “use as:” does not have an option for Reserved BIOS boot area. So, I booted into expert install mode, when you get to “Load installed components from CD” select parted. This will install parted in the install environment. Before you get to detect disks, do ctrl+alt+f2. On the command line you can manually create a bios boot area.
The following parted commands.
parted -a optimal /dev/sda mkpart 1 1 6
The above command creates the first (1) partition from 1MB of the drive to 6MB. -a optimal sets the block alignment for best performance. If you start the partition at 0 the alignment is wrong and parted will Warn: “The resulting partition is not properly aligned for best performance. Ignore/Cancel?”
parted /dev/sda set 1 bios_grub on
This sets /dev/sda1 as GPT grub bios partition. This partition will be found and used by grub on install.
Now, ctrl+alt+f1, and continue the install. Select manual partitioning and be sure not to delete the primary partition (gpt, grub bios) when creating your new partitions.
Do this to all drives in a raid!
After you boot into the fresh install, you can manually install to the other disks.
grub-install /dev/sdb
Now its installed on sda and sdb. In case sda fails, it should be able to boot from sdb.
At the bottom of gmail’s instructions you will see “recommended settings“. Don’t follow the first recommendation, regarding deleting. Instead of actually (permanently) deleting email we can have it moved to the Trash folder, as if you were using the gmail web interface. Gmail has strange folders structure in imap. Here are a few steps to work with gmail folders (Of course, they call them labels :-)
Under Account settings -> Copies and Folders
1. un-check “place a copy in… sent”
2. change “drafts and templates” to Other -> [your gmail address] -> [Gmail] -> Drafts Note: Sent and Drafts will automatically show up in their respective folders, as long as you send mail through smtp.gmail.com.
3. Go to Options/Preferences (under tools or edit menu). Go to Advanced -> Config Editor. In the Filter box, type “mail.server.server” and look for the gmail address. Whichever number is in the X place of mail.server.serverX, is what you want to use for the new value.
Right click New -> String “enter mail.server.serverX.trash_folder_name” (X is the number that correlates to your gmail account, from above). Next enter “[Gmail]/Trash”. Restart Thunderbird. Delete something, if it does not go into the [Gmail]/Trash folder restart again. It should work after that. Now you can delete the Trash folder (represented as [Imap]/Trash in gmail)
Contact syncing: A few options.
A very popular and stable add-on Zindus Its great but does not support the address field syncing. (Enter your user/pass under Tools -> Zindus) gContactSync has support for the address field but is very early in development. If any contacts have an empty email field it adds a @nowhere.invalid email address in Tbird. Thankfully this is not synced to your gmail account!
Also, This looks promising but does not work with Tbird3 Google Contacts.
BACKUP CONTACTS BEFORE SYNCING!
Calendar Sharing: Use the add-on Lightning (aka sunbird as a Tbird add on). Lightning 1.0b1 does not integrate with Thunderbird 3 toolbar layout very well. It messed up the “file” “edit” and “view” menus. Although, they aren’t really missed by anyone and are sort of accessible if you really need them. Try it for yourself, its not too bad.
For native support follow the Google sunbird instructions for CalDav (recommended)
There is also another plugin called Provider for Google Calendar that can be used..
look through the list of subkeys (0000, 0001, etc) DriverDesc keyword will tell you which network adapter. For example, “NVIDIA nForce Networking Controller”
when you find it, right click and add -> new -> string value
new keyword:
Value Name: = NetworkAddress
Value Data: = your new MAC, with no space or : For example, 0019DB71C830
then type:
ipconfig /release
ipconfig /renew
ipconfig /all (to verify the new mac took)
Now your a l33t Winblows H4|<3r! If you want to go back to your hardware MAC, remove the NetworkAddress key and restart the interface.
FYI, in gnu/linux follow these simple steps to change your MAC address
ifconfig eth0 hw ether 00:19:DB:71:C8:30
if you get this error
SIOCSIFHWADDR: Device or resource busy - you may need to down the interface
I wanted to share some notes on patching the Linux Libre kernel with realtime capabilities. The Linux-Libre project pulls out all the un-free bits from standard Linux. Contrary to popular belief, Linux has many non-free parts, small binary or obfuscated pieces of code for various hardware. I have a Lenovo T61 laptop. I removed the Intel wireless pci express card and put in a Atheros AR5008 wifi card using ath9k completely free wireless driver. Now my system (as far as I can tell:-) is completely free.
I make music and the realtime patch makes the latency of my system and soundcard very low. This is a unique advantage that the gnu/linux operating system gives its users. I highly recommend a realtime patch for anyone working with audio and video on gnu/linux.
tar xfvj linux-2.6.29.6-libre1.tar.bz2
cd linux-2.6.29.6
bzcat ../patch-2.6.29.6-rt23.bz2 | patch -p1
Now Linux is patched with realtime
now its time for
make menuconfig
from the RT How to:
* enable CONFIG_PREEMPT_RT
* activated the High-Resolution-Timer Option (Attention, the amount of supported platforms by the HR timer is still very limited. Right now the option is only supported on x86 systems, PowerPC and ARM Support are however in queue.)
* disabled all Power Management Options like ACPI or APM (not all ACPI functions are “bad”, but you will have to check very carefully to find out which function will affect your real time system. Thus it’s better to simply disable them all if you don’t need them. APM, however, is a no-go.) NOTE: Since rt patch 2.6.18-rt6 you will probably have to activate ACPI option to activate high resolution timer. Since the TSC timer on PC platforms, as used in the previous versions, are now marked as unsuitable for hrt mode due to many lacks of functionalities and reliabilties, you will need i.E. pm_timer as provided by ACPI to use as clock source. To activate the pm_timer, you can just activate the ACPI_SUPPORT in menuconfig and deactivate all other sub modules like “fan”, “processor” or “button”. If you have an old pc, which lacks ACPI support, you migh have problems using the high resolution timer.
I personally have not removed my power management options, as I use a laptop and want these features. I don’t notice any problems but have not tried it without them to know what I’m missing.