Saturday, July 12, 2008

BackupPC on FreeNAS

Please see also BackupPC on FreeNAS - Part 2 and BackupPC on FreeNAS - Part 3 (Issues with Samba)...


As I am a big fan of BackupPC, I'd like to have this piece of software on my FreeNAS-Server. Here is the way how I did the installation.


As I need some help to improve this way how I've done the installation... Please leave me a comment... 


BTW: This more than a alpha-stadium for this instructions.


I am using a full installation of FreeNAS 0.7. I like ZFS ;-)


It is necessary to install several Packages


freenas:~# setenv PACKAGESITE ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/YOUR_ARCHITECTURE/packages-7.0-release/

freenas:~# pkg_add -r perl5/perl-5.8.8_1

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/perl5/perl-5.8.8_1.tbz... Done.

Removing stale symlinks from /usr/bin...

    Skipping /usr/bin/perl

    Skipping /usr/bin/perl5

Done.

Creating various symlinks in /usr/bin...

    Symlinking /usr/local/bin/perl5.8.8 to /usr/bin/perl

    Symlinking /usr/local/bin/perl5.8.8 to /usr/bin/perl5

Done.

cd: can't cd to /usr/include

Cleaning up /etc/make.conf... Done.

Spamming /etc/make.conf... Done.

Cleaning up /etc/manpath.config... Done.

Spamming /etc/manpath.config... Done.

freenas:~# pkg_add -r perl5/p5-Compress-Raw-Zlib-2.006

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/perl5/p5-Compress-Raw-Zlib-2.006.tbz... Done.

freenas:~# pkg_add -r perl5/p5-Archive-Zip-1.20

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/perl5/p5-Archive-Zip-1.20.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/p5-File-Which-0.05.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/p5-Scalar-List-Utils-1.19,1.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/p5-PathTools-3.25.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/p5-File-Temp-0.18.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/p5-IO-Compress-Base-2.006.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/p5-IO-Compress-Zlib-2.006.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/p5-Compress-Zlib-2.007.tbz... Done.

freenas:~# pkg_add -r perl5/p5-File-RsyncP-0.68

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/perl5/p5-File-RsyncP-0.68.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/p5-Getopt-Long-2.36.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/p5-Encode-2.23.tbz... Done.

freenas:~# pkg_add -r net/samba-libsmbclient-3.0.28

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/net/samba-libsmbclient-3.0.28.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/libiconv-1.11_1.tbz... Done.

freenas:~# pkg_add -r net/samba-nmblookup-3.0.28.tbz

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/net/samba-nmblookup-3.0.28.tbz... Done.

freenas:~# pkg_add -r net/samba-3.0.28

Error: FTP Unable to get ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/net/samba-3.0.28.tbz: File unavailable (e.g., file not found, no access)

pkg_add: unable to fetch 'ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/net/samba-3.0.28.tbz' by URL

freenas:~# pkg_add -r net/samba-3.0.28,1

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/net/samba-3.0.28,1.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/openldap-client-2.3.41.tbz... Done.


************************************************************


The OpenLDAP client package has been successfully installed.


Edit

  /usr/local/etc/openldap/ldap.conf

to change the system-wide client defaults.


Try `man ldap.conf' and visit the OpenLDAP FAQ-O-Matic at

  http://www.OpenLDAP.org/faq/index.cgi?file=3

for more information.


************************************************************


Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/png-1.2.22.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/jpeg-6b_4.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/tiff-3.8.2_1.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/pkg-config-0.22_1.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/libexecinfo-1.1_1.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/gettext-0.16.1_3.tbz... Done.

install-info: not found

pkg_add: command 'install-info --quiet /usr/local/info/autosprintf.info /usr/local/info/dir' failed

install-info: not found

pkg_add: command 'install-info --quiet /usr/local/info/gettext.info /usr/local/info/dir' failed

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/popt-1.7_4.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/libgpg-error-1.5.tbz... Done.

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/libgcrypt-1.2.4_1.tbz... Done.

install-info: not found

pkg_add: command 'install-info --quiet /usr/local/info/gcrypt.info /usr/local/info/dir' failed

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/gnutls-2.0.2_1.tbz... Done.

install-info: not found

pkg_add: command 'install-info --quiet /usr/local/info/gnutls.info /usr/local/info/dir' failed

Fetching ftp://ftp.de.freebsd.org/pub/FreeBSD/ports/i386/packages-7.0-release/All/cups-base-1.3.5_2.tbz... Done.


======================================================================


To enable printing with local printer you need to give group cups

r/w access to printer device:


1) Add following to /etc/devfs.rules (create if it doesn't exist):


[system=10]

add path 'unlpt*' mode 0660 group cups

add path 'ulpt*' mode 0660 group cups

add path 'lpt*' mode 0660 group cups


2) And following to /etc/rc.conf:


devfs_system_ruleset="system"


3) Restart devfs: /etc/rc.d/devfs restart


In case your system supoorts "devd" you can copy

$PREFIX/share/exmaples/lpt-cupsd.conf into $PREFIX/etc/devd/



To enable printing under Gimp and MS-Windows clients do the following:


1) Uncomment application/octet-stream line in mime.types

2) Uncomment application/octet-stream line in mime.convs

3) Restart cupsd

======================================================================



===============================================================================

NOTICE: This version of port has changed location of Samba password

NOTICE: (smbpasswd) directory. Files in '/usr/local/private'

NOTICE: have moved to '/usr/local/etc/samba'.

===============================================================================

Samba3 *package* now doesn't include ADS support due the portability problems

with Kerberos5 libraries on different installations. You need to compile port

yourself to get this functionality.


For additional hints and directions, please, look into the README.FreeBSD file.

===============================================================================


Check the versions of tar and rsync


freenas:~# tar --version

bsdtar 2.2.5 - libarchive 2.2.4

freenas:~# rsync --version

rsync  version 3.0.2  protocol version 30

Copyright (C) 1996-2008 by Andrew Tridgell, Wayne Davison, and others.

Web site: http://rsync.samba.org/

Capabilities:

    64-bit files, 32-bit inums, 32-bit timestamps, 64-bit long ints,

    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,

    append, ACLs, xattrs, iconv, symtimes, file-flags


rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you

are welcome to redistribute it under certain conditions.  See the GNU

General Public Licence for details.


Create a filesystem where backuppc can store all the data


freenas:~# zfs create datapool/backuppc


Download BackupPC here


Upload the software to a directory on the FreeNAS Server 


Installation of BackupPC


Please see the answers below...


freenas:/mnt/space/backuppc/BackupPC-3.1.0# mkdir /usr/local/www

freenas:/mnt/space/backuppc/BackupPC-3.1.0# perl configure.pl


Is this a new installation or upgrade for BackupPC?  If this is

an upgrade please tell me the full path of the existing BackupPC

configuration file (eg: /etc/BackupPC/config.pl).  Otherwise, just

hit return.


--> Full path to existing main config.pl []? 


I found the following locations for these programs:


    bzip2        => 

    cat          => /bin/cat

    df           => /bin/df

    gtar/tar     => /usr/bin/tar

    gzip         => /usr/bin/gzip

    hostname     => /bin/hostname

    nmblookup    => /usr/local/bin/nmblookup

    par2         => 

    perl         => /usr/bin/perl

    ping         => /sbin/ping

    rsync        => /usr/local/bin/rsync

    sendmail     => 

    smbclient    => /usr/local/bin/smbclient

    split        => 

    ssh/ssh2     => /usr/bin/ssh


--> Are these paths correct? [y]? 


Please tell me the hostname of the machine that BackupPC will run on.


--> BackupPC will run on host [freenas.harrys.homelinux.net]? 


BackupPC should run as a dedicated user with limited privileges.  You

need to create a user.  This user will need read/write permission on

the main data directory and read/execute permission on the install

directory (these directories will be setup shortly).


The primary group for this user should also be chosen carefully.

The data directories and files will have group read permission,

so group members can access backup files.


--> BackupPC should run as user [backuppc]? root


Please specify an install directory for BackupPC.  This is where the

BackupPC scripts, library and documentation will be installed.


--> Install directory (full path) [/usr/local/BackupPC]? 


Please specify a data directory for BackupPC.  This is where all the

PC backup data is stored.  This file system needs to be big enough to

accommodate all the PCs you expect to backup (eg: at least several GB

per machine).


--> Data directory (full path) [/data/BackupPC]? /mnt/datapool/backuppc


BackupPC can compress pool files, providing around a 40% reduction in pool

size (your mileage may vary). Specify the compression level (0 turns

off compression, and 1 to 9 represent good/fastest to best/slowest).

The recommended values are 0 (off) or 3 (reasonable compression and speed).

Increasing the compression level to 5 will use around 20% more cpu time

and give perhaps 2-3% more compression.


--> Compression level [3]? 


BackupPC has a powerful CGI perl interface that runs under Apache.

A single executable needs to be installed in a cgi-bin directory.

This executable needs to run as set-uid root, or

it can be run under mod_perl with Apache running as user root.


Leave this path empty if you don't want to install the CGI interface.


--> CGI bin directory (full path) []? /usr/local/www


BackupPC's CGI script needs to display various GIF images that

should be stored where Apache can serve them.  They should be

placed somewhere under Apache's DocumentRoot.  BackupPC also

needs to know the URL to access these images.  Example:


    Apache image directory:  /usr/local/apache/htdocs/BackupPC

    URL for image directory: /BackupPC


The URL for the image directory should start with a slash.


--> Apache image directory (full path) []? /usr/local/www/images

--> URL for image directory (omit http://host; starts with '/') []? /images


Ok, we're about to:


  - install the binaries, lib and docs in /usr/local/BackupPC,

  - create the data directory /mnt/datapool/backuppc,

  - create/update the config.pl file /etc/BackupPC/config.pl,

  - optionally install the cgi-bin interface.


--> Do you want to continue? [y]? 

Created /usr/local/BackupPC/bin

Created /usr/local/BackupPC/doc

Created /usr/local/BackupPC/lib/BackupPC/CGI

Created /usr/local/BackupPC/lib/BackupPC/Config

Created /usr/local/BackupPC/lib/BackupPC/Lang

Created /usr/local/BackupPC/lib/BackupPC/Storage

Created /usr/local/BackupPC/lib/BackupPC/Xfer

Created /usr/local/BackupPC/lib/BackupPC/Zip

Created /usr/local/www/images

Created /mnt/datapool/backuppc

Created /mnt/datapool/backuppc/pool

Created /mnt/datapool/backuppc/cpool

Created /mnt/datapool/backuppc/pc

Created /mnt/datapool/backuppc/trash

Created /etc/BackupPC

Created /var/log/BackupPC

Installing binaries in /usr/local/BackupPC/bin

Installing library in /usr/local/BackupPC/lib

Installing images in /usr/local/www/images

Making init.d scripts

Installing docs in /usr/local/BackupPC/doc

Installing config.pl and hosts in /etc/BackupPC

Installing cgi script BackupPC_Admin in /usr/local/www


Ok, it looks like we are finished.  There are several more things you

will need to do:


  - Browse through the config file, /etc/BackupPC/config.pl,

    and make sure all the settings are correct.  In particular,

    you will need to set $Conf{CgiAdminUsers} so you have

    administration privileges in the CGI interface.


  - Edit the list of hosts to backup in /etc/BackupPC/hosts.


  - Read the documentation in /usr/local/BackupPC/doc/BackupPC.html.

    Please pay special attention to the security section.


  - Verify that the CGI script BackupPC_Admin runs correctly.  You might

    need to change the permissions or group ownership of BackupPC_Admin.

    If this is an upgrade and you are using mod_perl, you will need

    to restart Apache.  Otherwise it will have stale code.


  - BackupPC should be ready to start.  Don't forget to run it

    as user root!  The installation also contains an

    init.d/backuppc script that can be copied to /etc/init.d

    so that BackupPC can auto-start on boot.  This will also enable

    administrative users to start the server from the CGI interface.

    See init.d/README.


Enjoy!


It is necessary to change one setting in /var/etc/lighttpd.conf


Save the original file

freenas:/var/etc# cd /var/etc/

freenas:/var/etc# cp lighttpd.conf lighttpd.conf.orig

I have no idea how to make this setting persistent. After a reboot the change is lost...


This is what didn't work... BackupPC works like a charm, but it is not possible to use the web interface after a reboot.


Change this row


cgi.assign = (".php" => "/usr/local/bin/php")


to this


cgi.assign = (".php" => "/usr/local/bin/php",

"/BackupPC_Admin" => "")


Copy the Startup-Scripts to /usr/local/etc/rc.d


freenas:/mnt/space/backuppc/BackupPC-3.1.0# cp init.d/freebsd-backuppc /usr/local/etc/rc.d/backuppc


Set permissions in /usr/local/etc/rc.d/


I don't want that cupsd or samba will be started from here...


freenas:/mnt/space/backuppc/BackupPC-3.1.0# cd /usr/local/etc/rc.d/

freenas:/usr/local/etc/rc.d# chmod 644 samba 

freenas:/usr/local/etc/rc.d# chmod 644 cupsd 



Change a setting in /usr/local/etc/rc.d/backuppc


    su backuppc -c '/usr/local/BackupPC/bin/BackupPC -d'


in


    su root -c '/usr/local/BackupPC/bin/BackupPC -d'


Add the following to /etc/rc.conf


backuppc_enable="YES"


19 comments:

Pablo said...

I followed all steps, but when i look for http://local_IP_freenas/BackupPC_Admin ... the browser offer me to download the script, instead of run it on server.

:(

harryd said...

Have you edited /var/etc/lighttpd.conf ?
Try to restart lighty (find out the process with ps axuw, kill it and start it again)

Pablo said...

Yes i tried. Now i've got this:

http://micromed.vc/arquivos/Picture2.png

Pablo said...

I think the problem is in lighttpd.conf

You said:

This is what didn't work... BackupPC works like a charm, but it is not possible to use the web interface after a reboot.


Change this row


cgi.assign = (".php" => "/usr/local/bin/php")


to this


cgi.assign = (".php" => "/usr/local/bin/php",

"/BackupPC_Admin" => "")


and this is a little different from those intructions

http://backuppc.wiki.sourceforge.net/Install+Backuppc+with+Lighttpd

what can i do ?

harryd said...

On your screenshot I can see that the BackupPC_Admin script is running correctly. Right?

Maybe you also need to edit this /etc/BackupPC/config.pl

$Conf{CgiAdminUserGroup} = '';
$Conf{CgiAdminUsers} = '*';

The problem with the /var/etc/lighttpd.conf is, that the changes will be lost after a reboot. You are right, my setting in lighttpd.conf is different, but it works for me...

If you want you can reach me via harryd71@users.sourceforge.net

Anonymous said...

This is useful as I always wanted this to run on FreeBSD but could never quite get it to work.

My only concern is the use of zfs which is not really stable yet and as backups should be a conservitive setup I think I'll stick to ufs.

Thanks for posting the how to now if someone could just build a port/package :)

Unknown said...

I just finished installing backuppc on my freenas. thanks for the awesome post.

I figured out a way to workaround the lighttpd.conf issue.

In the backuppc_start() method inside of /usr/local/etc/rc.d/backuppc file, I added the following:

kill -9 `ps auxw | grep light | grep -v grep | awk '{print $2}'`

lighttpd -f /var/etc/lighttpd_backuppc.conf start

where lighttpd_backuppc.conf contains the /BackupPC_Admin argument for cgi.assign.

I rebooted my machine with these changes and it works!

harryd said...

Thanks John for your workaround! Maybe you've seen my workaround here -> http://harryd71.blogspot.com/2008/07/backuppc-on-freenas-part-2.html this works also after a reboot.

Michael said...

This has been a good basis for my own exploration of FreeNAS/BackupPC, but I have a question. I've got my set rigged out to use the native Webserver function of FreeNAS on port 8080 with authentication and SSL, which is what is needed for it to work well in our context. Everything works well, apart from the fact that I am loathe to run Backuppc as the root user. I've tried running the server and telling it to use a different user for rsync purposes, but that does not work. Do you have any insight in getting Backuppc to run under a non-root user? I've installed as 'backuppc', but as you may be aware, the script vomits up a uid warning, saying that it is running under uid 0 instead of xxxx. I set user under FreeNAS in wheel and tinkered with permissions, but no luck.

harryd said...

@Michael
Unfortunately I've only set up my backuppc to run as a root user. So, no I don't have any experience, sorry...

Michael said...

No problem, I'll keep plunking about until I get it to work. If there is a breakthrough, I'll lay out some details here as it's the touchstone thread for this sort of thing.

Anonymous said...

Sorry for the delay. Right now I have it working the way I want to. All rsync commands are run as user "backuppc" ($sshPath -q -x -l backuppc $hostIP sudo $rsyncPath), while the server is run as user "root". My mix up was to first use the "backuppc" users public/private key for making a connection to the client, which fails as it is initiated by "root". Using roots public/private key pair resolved the issue. I've had BackupPC running over FreeNAS for four months now without a hitch.

Unknown said...

Hi !

Thank you for your help, I think to use BackupPC with my FreeNAS server will be very usefully for me.

I done all changes in my FreeNAS server, but I try to change my lighttpd.conf file and I change it without problems, restart my lighttpd deamon, and I can't to access to the new configuration and when I watch again the config file (lighttpd.con) I see that any changes are there, but they're there when a save the file.!!!!

Could you know any tip about it?

thank's in advance.

Luis

harryd said...

Do you've tried John's tip?

Unknown said...

Hi! again

Thank you for your help, I used the John's workaround without success.

I checked with "ps -auxw|grep backuppc" command and I haven't running any backuppc daemon.

I'll try again to install all from the begin, and then I comment the result.

Thank's again.

Anonymous said...

Seems to have most thing working by now, but when i try a full backup, the following error shows up in the LOG:
Can't locate File/Listing.pm in @INC.
Do I have to install another package?

harryd said...

@Anonymous: I haven't used BackupPC for a long time. It looks like you have to install the File::Listing module.

Anonymous said...

Has you done this on freenas-0.8 yet?

harryd said...

@Anonymous Sorry... No, I haven't. I am currently pretty busy. At the moment I use a virtual machine on an ESX Server as backuppc. I have not the 'requirement' to run backuppc on freenas...