Setup Local Repositories with ‘apt-mirror’ in Ubuntu and Debian Systems

When today traffic and casual internet speeds is measured in teens of Giga over an eye blink even for ordinary Internet clients, what’s the purpose of setting a local repository cache on LAN’s you may ask?

Setup Local Repositories in Ubuntu

Setup Local Repositories in Ubuntu

One of the reasons is to reduce Internet bandwidth and high speed on pulling packages from local cache. But, also, another major reason should be privacy. Let’s imagine that clients from your organization are Internet restricted, but their Linux boxes need to regular system updates on software and security or just need new software packages. To go further picture, a server that runs on a private network, contains and serves secret sensitive information only for a restricted network segment, and should never be exposed to public Internet.

This are just a few reasons why you should build a local repository mirror on your LAN, delegate an edge server for this job and configure internal clients to pull out software form its cache mirror.

Ubuntu provides apt-mirror package to synchronize local cache with official Ubuntu repositories, mirror that can be configured through a HTTP or FTP server to share its software packages with local system clients.

For a complete mirror cache your server needs at least 120G free space reserved for local repositories.

Requirements

  1. Min 120G free space
  2. Proftpd server installed and configured in anonymous mode.

Step 1: Configure Server

1. The first thing you may want to do is to identify the closest and fastest Ubuntu mirrors near you’re location by visiting Ubuntu Archive Mirror page and select your country.

Ubuntu Archive Mirror

Ubuntu Archive Mirror

If your country provides more mirrors you should identify mirror address and do some tests based on ping or traceroute results.

Select Mirror Location

Select Mirror Location

2. The next step is to install required software for setting up local mirror repository. Install apt-mirror and proftpd packages and configure proftpd as standalone system daemon.

$ sudo apt-get install apt-mirror proftpd-basic
Install apt-mirror Proftpd

Install apt-mirror Proftpd

ProFTPD Configuration

ProFTPD Configuration

3. Now it’s time to configure apt-mirror server. Open and edit /etc/apt/mirror.list file by adding your nearest locations (Step 1) – optional, if default mirrors are fast enough or you’re not in a hurry – and choose your system path where packages should be downloaded. By default apt-mirror uses /var/spool/apt-mirror location for local cache but on this tutorial we are going to use change system path and point set base_path directive to /opt/apt-mirror location.

$ sudo nano /etc/apt/mirror.list
Configure apt-mirror Server.

Configure apt-mirror Server.

Also you can uncomment or add other source list before clean directive – including Debian sources – depending on what Ubuntu versions your clients use. You can add sources from 12.04, if you like but be aware that adding more sources requires more free space.

For Debian source lists visit Debian Wiki or Debian Sources List Generator.

4. All you need to do now is, just create path directory and run apt-mirror command to synchronize official Ubuntu repositories with our local mirror.

$ sudo mkdir -p /opt/apt-mirror
$ sudo apt-mirror
Create apt-mirror Paths

Create apt-mirror Paths

As you can see apt-mirror proceeds with indexing and downloading archives presenting total number of downloaded packages and their size. As we can imagine 110-120 GB is large enough to take some time to download.

You can run ls command to view directory content.

Verify apt-mirror Paths

Verify apt-mirror Paths

Once the initial download is completed, future downloads will be small.

5. While apt-mirror downloads packages, you can configure your Proftpd server. The first thing you need to do is, to create anonymous configuration file for proftpd by running the following command.

$ sudo nano /etc/proftpd/conf.d/anonymous.conf

Then add the following content to anonymous.conf file and restart proftd service.

<Anonymous ~ftp>
   User                    ftp
   Group                nogroup
   UserAlias         anonymous ftp
   RequireValidShell        off
#   MaxClients                   10
   <Directory *>
     <Limit WRITE>
       DenyAll
     </Limit>
   </Directory>
 </Anonymous>
Configure ProFTPD

Configure ProFTPD

6. Next step is to link apt-mirror path to proftpd path by running a bind mount by issuing the command.

$ sudo mount --bind /opt/apt-mirror/mirror/archive.ubuntu.com/  /srv/ftp/
Mount apt-mirror to ProFTP Path

Mount apt-mirror to ProFTP Path

To verify it run mount command with no parameter or option.

$ mount
Verify Paths

Verify Paths

7. Last step is to make sure that Proftpd server is automatically started after system reboot and mirror-cache directory is also automatically mounted on ftp server path. To automatically enable proftpd run the following command.

$ sudo update-rc.d proftpd enable

To automatically mount apt-mirror cache on proftpd open and edit /etc/rc.local file.

$ sudo nano /etc/rc.local

Add the following line before exit 0 directive. Also use 5 seconds delay before attempting to mount.

sleep 5
sudo mount --bind  /opt/apt-mirror/mirror/archive.ubuntu.com/ /srv/ftp/
Auto Mount Apt Mirrors

Auto Mount Apt Mirrors

If you pull packages from Debian repositories run the following commands and make sure appropriate settings for above rc.local file are enabled.

$ sudo mkdir /srv/ftp/debian
$ sudo mount --bind /opt/apt-mirror/mirror/ftp.us.debian.org/debian/ /srv/ftp/debian/
Debian Repository Setup

Debian Repository Setup

8. For a daily apt-mirror synchronization you can also create a system schedule job to run at 2 AM every day. Run crontab command, select your preferred editor then add the following line syntax.

$ sudo crontab –e
Daily apt-mirror Synchronization

Daily apt-mirror Synchronization

On last line add the following line.

0  2  *  *  *  /usr/bin/apt-mirror >> /opt/apt-mirror/mirror/archive.ubuntu.com/ubuntu/apt-mirror.log
Add Cron Entry for Synchronization

Add Cron Entry for Synchronization

Now every day at 2 AM your system repository cache will synchronize with Ubuntu official mirrors and create a log file.

Step 2: Configure clients

9. To configure local Ubuntu clients, edit /etc/apt/source.list on client computers to point to the IP address or hostname of apt-mirror server – replace http protocol with ftp, then update system.

deb ftp://192.168.1.13/ubuntu trusty universe
deb ftp://192.168.1.13/ubuntu trusty main restricted
deb ftp://192.168.1.13/ubuntu trusty-updates main restricted
## Ad so on….
Configure Clients

Configure Clients

10. To view repositories you can actually open a browser and point to your server IP address of domain name using FTP protocol.

View Local Repositories

View Local Repositories

The same system applies also to Debian clients and servers, the only change needed are debian mirror and sources list.

Also if you install a fresh Ubuntu or Debian system, provide your local mirror manually whit ftp protocol when installer asks which repository to use.

The great thing about having your own local mirror repositories is that you’re always on current and your local clients don’t have to connect to Internet to install updates or software.

If You Appreciate What We Do Here On TecMint, You Should Consider:

TecMint is the fastest growing and most trusted community site for any kind of Linux Articles, Guides and Books on the web. Millions of people visit TecMint! to search or browse the thousands of published articles available FREELY to all.

If you like what you are reading, please consider buying us a coffee ( or 2 ) as a token of appreciation.

Support Us

We are thankful for your never ending support.

Matei Cezar

I'am a computer addicted guy, a fan of open source and linux based system software, have about 4 years experience with Linux distributions desktop, servers and bash scripting.

Your name can also be listed here. Got a tip? Submit it here to become an TecMint author.

RedHat RHCE and RHCSA Certification Book
Linux Foundation LFCS and LFCE Certification Preparation Guide

You may also like...

13 Responses

  1. Ravindra Maghudiya says:

    hi Matei Cezar, thanks a lot this great tutorial, i am all done with this process and now my network repository running very well.
    but i want too add some extra package other then default mint and ubuntu repository like google chrome,wpa office, in short i want the repository in that i can add my own deb or tar package and install it in my network. waiting for your great response, thanks in advance.

    • Matei Cezar says:

      I do not recommend adding thirty-party software in apt-mirror list, especially non-free software. Add only repos maintained by the distribution. As far as a know Kingsoft Office doesn’t provide any ppa for Ubuntu, only deb packages. Try to download the latest deb packages for the applications you need and place them in a new directory on the ftp server (/srv/ftp/my_deb for example).

  2. Luc Filiatrault says:

    I have 2 Debian client and one Debian server (back-up server). The later took less than 20 minutes to pass from Wheezy to Jessie .
    Do you think I would save bandwith if I build a repository basically for the 2 clients,considering when I move from one version to the next I will have to download all the new version packages again.

    Cheers,

    • Ravi Saive says:

      @Luc,
      If you setup a local repository in one of your sever for 2 clients, it will save bandwidth as it downloads and install packages from your local repository..

      In short, your local repository server will first download all packages in your server and then clients will fetch packages from your repository server, this way you can save bandwidth and time..

  3. hatta says:

    thank you for the information, I’ve tried to configure the server and succeed. but when I try to access via ftp client, it appears username and password. my question is how to fix it ?

    • Vimal says:

      restart proftpd service. it should work now

      • sneha says:

        For me also same issue now when I restarted the service I am getting site cant be reached,Please help how to solve this?

        • Matei Cezar says:

          Check if the ftp service is up and running and the corresponding ports are opened in firewall.

          • sneha says:

            Thanks for replying, now when I restarted the ftp service, service is up but now my ftp://public ip is not loading.Please let me know in aws which ports I need to open.I opened port 20 and 21. But no result.

          • Matei Cezar says:

            You should open ftp passive incoming ports. Consult ftp conf file fo the passive port range you need to open on aws for your ftp server (ports should be mentined there) or just open all incoming range from 1024 to 65534. Also, check ftp://localhost to test if is working or use a ssh tunnel for ftp service.

          • Matei Cezar says:

            20,21 Incoming ports opened, 1024-65534 (or just ftp range) for outgoing.

  4. Joe says:

    I ran apt-mirror and it downloaded a ton of stuff, so far so good, but it only created the ubuntu/pool directory, I don’t have necessary directories such as ubuntu/dists, and this causes the client to fail apt-get update. Any idea what would cause that and how to fix it?

  5. Etescartz says:

    Thanks for the great reading material!
    I haven’t had the chance to test your setup in production but I do have one set up with nginx as a web server repository. It used by just about 1000 clients and it is really fast.
    The hardware has more of a 2003-2004 feeling about it but it does the job flawlessly.

Leave a Reply to hatta Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.