jason schaefer . com

"arguing that you don’t care about the right to privacy because you have nothing to hide is no different than saying you don’t care about free speech because you have nothing to say."

Category: backup

Migrating SMSSecure (Silence) keys and texts
For those unfamiliar: SMSSecure (now called Silence) is a text messaging application that supports encryption. It has exceptional integration on your phone and will handle your non-encrypted ~~friends~~ acquaintances texts as well.
https://smssecure.org
I recommend using fdroid to install it. F-droid is a Free Software repository for Android.
https://f-droid.org

UPDATE: You can now export an encrypted copy of Silence. This export includes your encryption keys. From your old phones Silence “app”lication, under the three horizontal “…” -> “Import / Export” -> “Export encrypted backup”. This exports to the phones /sdcard/SilenceExport or from mtp “mtp://[usb:xxx,xxx]/Internal storage/SilenceExport”. Copy this directory to the new phone, to the equivalent path (/sdcard/). Install Silence on the new phone. In my experience the first-run import prompt didn’t work so I had to do it manually. Click the three horizontal “…” -> “Import / Export” -> “Restore encrypted backup”. Your done!

I wanted to do this transfer using mtp but neither thunar or nautilus could “see” the SilenceExport directory. I had to do the following:
plug in old phone and copy the export to my laptop:
adb pull /sdcard/SilenceExport/
plug in the new phone:
adb push SilenceExport/
Remember you have to authorize your laptop from the phone.

THE FOLLOWING INSTRUCTIONS ARE CONSIDERED OUT OF DATE yet still useful for some.

When moving to a new phone you can export a plain text copy of all your sms texts for import on the new phone. This is fine but has many drawbacks. ~~The export feature of smssecure does not export a users encryption keys.~~ Here are instructions on doing this yourself. Its really not as bad as it might seem and is worth doing to avoid re-keying with every friend you text with. You can also avoid exporting sensitive txt’s to plain-text.

PREREQ’s:
Install SMSSecure on the new phone.
You will need adb access to both phones.

Enable developer options by going into Settings -> About phone -> then press “Build number” 7 times.

Enable adb access under Settings -> Developer options -> Android debugging.

Allow adb root access from “developer options -> Root access” “Apps and ADB”. Otherwise you will get a permission error like so:
```
shell@ville:/ $ su -
Permission denied
```
Also, be sure your computer can see the phone:
[jason@local ~] $ adb devices List of devices attached 393520931D5B00EC device
Now connect to the phone over the network:
[jason@local ~] $ adb connect 192.168.123.12:5555

You can continue as if it was plugged in directly…
To disconnect from the phone use this:
[jason@local ~] $ adb disconnect 192.168.123.12:5555

STEPS:
I wasn’t able to directly copy the smssecure directory to my computer. It errors with zero files transferred:
```
[jason@local ~] $ adb pull /data/data/org.smssecure.smssecure/ org.smssecure.smssecure-balz
pull: building file list...
0 files pulled. 0 files skipped.
```
First I copy it to the sdcard of the old phone:
```
[jason@local ~] $ adb shell
shell@crespo:/ $ su -
root@crespo:/ # cp -rv /data/data/org.smssecure.smssecure/ /sdcard/
```
Copy the smssecure data from your old phone /sdcard/ to your local computer:
```
[jason@local ~] $ adb pull /sdcard/org.smssecure.smssecure/ org.smssecure.smssecure
```
Now plugin the new phone

Next, we will copy the org.smssecure.smssecure directory to the new phone:
Note: I attempted to copy directly to the new phone at /data/data but there was a permission denied:
```
[jason@local ~] $ adb push org.smssecure.smssecure/ /data/data/
push: org.smssecure.smssecure/databases/messages.db-journal -> /data/data/databases/messages.db-journal
failed to copy 'org.smssecure.smssecure/databases/messages.db-journal' to '/data/data/databases/messages.db-journal': Permission denied
```
Instead copy it to the /sdcard/ of the new phone, first:
UPDATE: This doesn’t work anymore
```
[jason@local ~] $ adb push org.smssecure.smssecure/ /sdcard/
adb: warning: skipping empty directory 'org.smssecure.smssecure/code_cache/'
adb: warning: skipping empty directory 'org.smssecure.smssecure/app_captures/'
adb: error: failed to copy 'org.smssecure.smssecure/lib' to '/sdcard/org.smssecure.smssecure/lib': symlink failed: Operation not permitted
```
The brilliant developers of adb didn’t write in support for adb push to be recursive? We have to get creative as usual with Android. Use zip if you have unzip on your phone.
```
[jason@local ~] $ zip -r org.smssecure.smssecure.zip org.smssecure.smssecure/

And then,
adb push org.smssecure.smssecure.zip /sdcard/
adb shell
cd /sdcard/
unzip org.smssecure.smssecure.zip
```
Before we proceed lets check what permissions the org.smssecure.smssecure directory is currently set to (from the phones shell). In this case its chown u0_a63.u0_a63 and chmod 755:
```
root@crespo:/ # ls -ld /data/data/org.smssecure.smssecure/
drwxr-xr-x u0_a63   u0_a63            2015-12-06 21:35
```
Keep note of this for later.

Using the new phone’s root shell you will delete the existing (un-used) directory and copy your smssecure directory (with all your text’s) from sdcard to /data/data:
```
[jason@local ~] $ adb shell
shell@crespo:/ $ su -
root@crespo:/ # 
root@crespo:/ # rm -fr /data/data/org.smssecure.smssecure/
root@crespo:/ # cp -rv /sdcard/org.smssecure.smssecure  /data/data/
```
Note: If you try using move (mv) instead of copy (cp) and your sdcard is on a separate partition you will get this error:
root@maguro:/ # mv /sdcard/org.smssecure.smssecure/ /data/data/ failed on '/sdcard/org.smssecure.smssecure/' - Cross-device link 255|

Back to permissions:
After the data is copied to the new phone the directory and file permissions will be wrong (owned by root):
```
root@crespo:/ # ls -l /data/data/org.smssecure.smssecure/       
drwxrwx--- root     root              2015-12-06 21:35 app_parts
drwxrwx--- root     root              2015-12-06 21:35 databases
drwxrwx--- root     root              2015-12-06 21:35 files
lrwxrwxrwx install  install           2015-12-06 15:07 lib -> /data/app-lib/org.smssecure.smssecure-1
drwxrwx--- root     root              2015-12-06 21:35 shared_prefs
```
We need to change ownership to user and group. In my case I need to change it to u0_a63. Android’s chown doesn’t work as expected:
```
root@crespo:/ # chown -R u0_a63.u0_a63 /data/data/org.smssecure.smssecure/
No such user '-R'
```
Unfortunately, android sucks and chown is broken so we must get creative. Like any puzzle, its simple once you know the answer :-)
```
root@crespo:/ # find /data/data/org.smssecure.smssecure/ -exec chown u0_a63.u0_a63 {} \;
```
Luckily, chmod works ok:
```
chmod -R 755 /data/data/org.smssecure.smssecure/
```
Be sure to reboot your phone.
Note: My SMSSecure disappeared and I needed to re-install. Everything was there after I re-installed and it survived subsequent reboots.

done;
December 6, 2015
SpamAssassin training and spam cleanup script
Spam is a constant battle as it is ever changing and always creeping into your Inbox. Spam wrangling is only effective with proper training, SpamAssassin does a decent job out-of-the-box but needs users input to truly be effective. This script will run SpamAssassin’s built in sa-learn tool against users known spam and known ham.

With my setup — A spam message is received (postfix) and identified as spam (spamassassin), it will be moved to the users Junk directory per the Sieve (dovecot) rule at the bottom of this post. If a spam message is received and not matched as such, it will be delivered to the Inbox. The user will identify it and manually move it to the Junk directory. I like to configure Thunderbird to “Move new junk messages to: “Junk” folder on: … in Account Settings.” Now a user marks a message as Junk, it automatically gets moved.

When this script runs it will mark messages in the Inbox as ham and messages in the Junk directory as spam. The command “sa-learn –sync” adds these results to the Bayes database that SpamAssassin consults when determining a spam score for each received message. This database is optionally backed up in the event a mistake was made and you need to revert back to previous versions. The script logs to /var/log/train-mail.log, information about how much spam and ham is being added, how many total messages have been processed and stats on the auto clean feature built into sa-learn can be gleaned. Additionally, I setup a spam cleanup using find and rm.

If sa-learn scans a mail as ham when it is spam, or vice versa, simply move the messages to the correct directory (Inbox=ham/Junk=spam), and the mistake will be corrected on the next run. SpamAssassin will automatically ‘forget’ the previous indications.

Its important to have an equal part ham to spam. As a result I run this script daily in an effort to capture users ham before they delete it or sort it into sub folders. Another thing to be aware of is that typically you should aim to train with at least 1000 messages of spam, and 1000 ham messages, if possible. More is better, but anything over about 5000 messages does not improve accuracy significantly in SpamAssassins tests.

Obviously a lot of these options are site/user specific. This is a good foundation to use as-is or build from.
```
#!/bin/bash

#specify one or more users, space padded [user=(user1 user2 user3)] or empty [user=()] to include all users. All users is considered uid ≥ 1000.
user=()

#After how many days should Spam be deleted?
cleanafter=30

#backup path, comment out to disable backups
bk=/home/backup/sa-learn_bayes_`date +%F`.backup

log=/var/log/train-mail.log
#log=/dev/stdout

echo -e "\n`date +%c`"  >> $log 2>&1

if [ -z ${user[@]} ]; then
echo user is empty, using all users from system
user=(`awk -F':' '$3 >= 1000 && $3 < 65534' /etc/passwd |awk -F':' '{print $1}'`)
fi

for u in ${user[@]}; do
if [ ! -d /home/$u/Maildir ]; then
echo "No such Maildir for $u" >> $log 2>&1
else
echo "Proceeding with ham and spam training on user \"$u\""
#add all messages in "junk" directory to spamassassin
echo spam >> $log 2>&1
#change this path to match your spam directory, in this case its "Junk"
#add current and new messages in Junk directory as spam
sa-learn --no-sync --spam /home/$u/Maildir/.Junk/{cur,new} >> $log 2>&1
echo ham >> $log 2>&1
#only add current mail to ham, not new. This gives user a chance to move it to spam dir.
sa-learn --no-sync --ham /home/$u/Maildir/{cur} >> $log 2>&1
fi
done

#sync the journal created above with the database
echo sync >> $log 2>&1
sa-learn --sync >> $log 2>&1
if [ $? -eq 0 ]; then
for u in ${user[@]}; do
echo "deleting spam for $u older than 30 days" >> $log 2>&1
find /home/$u/Maildir/.Junk/cur/ -type f -mtime +$cleanafter -exec rm {} \;
done
else
echo "sa-learn wasn't able to sync. Something is broken. Skipping spam cleanup"
fi

echo "Statistics:" >> $log 2>&1
sa-learn --dump magic >> $log 2>&1
echo ============================== >> $log 2>&1

if [ -n $bk ]; then
echo "backup writing to $bk" >> $log 2>&1
sa-learn --backup > $bk
fi
```
Here is my sieve rule for moving messages that are marked as spam to my Junk directory. I setup roundcube for people to manage their sieve filters.
```
$ cat /home/jason/sieve/managesieve.sieve
require ["fileinto"];
# rule:[SPAM]
if header :contains "X-Spam-Status" "Yes"
{
	fileinto "Junk";
	stop;
}
```
January 26, 2013
Backup Scripts in bash and batch

Here is a bash script that I use to do simple and transparent backups of all sorts of data.
I use it primarily to backup to multiple external hard drives, plugged in via USB. It uses regular mount or gnome volume manager. It also sends logs via email using “mail” and a local MTA like postfix. Please modify to fit your needs. Leave a comment if you want. Its also here http://jasonschaefer.com/stuff/jsbk.sh.txt

http://jasonschaefer.com/stuff/jsbk.sh.txt

Here is my batch script I use for MS Windows boxen. Batch scripting is nasty but surprisingly this script has come in handy many a time. Here is a direct link http://jasonschaefer.com/stuff/jsbkwin.bat.txt

http://jasonschaefer.com/stuff/jsbkwin.bat.txt

December 6, 2011