Setup Local Repositories with ‘apt-mirror’ in Ubuntu and Debian Systems

When today traffic and casual internet speeds is measured in teens of Giga over an eye blink even for ordinary Internet clients, what’s the purpose of setting a local repository cache on LAN’s you may ask?

Setup Local Repositories in Ubuntu
Setup Local Repositories in Ubuntu

One of the reasons is to reduce Internet bandwidth and high speed on pulling packages from local cache. But, also, another major reason should be privacy. Let’s imagine that clients from your organization are Internet restricted, but their Linux boxes need to regular system updates on software and security or just need new software packages. To go further picture, a server that runs on a private network, contains and serves secret sensitive information only for a restricted network segment, and should never be exposed to public Internet.

This are just a few reasons why you should build a local repository mirror on your LAN, delegate an edge server for this job and configure internal clients to pull out software form its cache mirror.

Ubuntu provides apt-mirror package to synchronize local cache with official Ubuntu repositories, mirror that can be configured through a HTTP or FTP server to share its software packages with local system clients.

For a complete mirror cache your server needs at least 120G free space reserved for local repositories.

Requirements

  1. Min 120G free space
  2. Proftpd server installed and configured in anonymous mode.

Step 1: Configure Server

1. The first thing you may want to do is to identify the closest and fastest Ubuntu mirrors near you’re location by visiting Ubuntu Archive Mirror page and select your country.

Ubuntu Archive Mirror
Ubuntu Archive Mirror

If your country provides more mirrors you should identify mirror address and do some tests based on ping or traceroute results.

Select Mirror Location
Select Mirror Location

2. The next step is to install required software for setting up local mirror repository. Install apt-mirror and proftpd packages and configure proftpd as standalone system daemon.

$ sudo apt-get install apt-mirror proftpd-basic
Install apt-mirror Proftpd
Install apt-mirror Proftpd
ProFTPD Configuration
ProFTPD Configuration

3. Now it’s time to configure apt-mirror server. Open and edit /etc/apt/mirror.list file by adding your nearest locations (Step 1) – optional, if default mirrors are fast enough or you’re not in a hurry – and choose your system path where packages should be downloaded. By default apt-mirror uses /var/spool/apt-mirror location for local cache but on this tutorial we are going to use change system path and point set base_path directive to /opt/apt-mirror location.

$ sudo nano /etc/apt/mirror.list
Configure apt-mirror Server.
Configure apt-mirror Server.

Also you can uncomment or add other source list before clean directive – including Debian sources – depending on what Ubuntu versions your clients use. You can add sources from 12.04, if you like but be aware that adding more sources requires more free space.

For Debian source lists visit Debian Wiki or Debian Sources List Generator.

4. All you need to do now is, just create path directory and run apt-mirror command to synchronize official Ubuntu repositories with our local mirror.

$ sudo mkdir -p /opt/apt-mirror
$ sudo apt-mirror
Create apt-mirror Paths
Create apt-mirror Paths

As you can see apt-mirror proceeds with indexing and downloading archives presenting total number of downloaded packages and their size. As we can imagine 110-120 GB is large enough to take some time to download.

You can run ls command to view directory content.

Verify apt-mirror Paths
Verify apt-mirror Paths

Once the initial download is completed, future downloads will be small.

5. While apt-mirror downloads packages, you can configure your Proftpd server. The first thing you need to do is, to create anonymous configuration file for proftpd by running the following command.

$ sudo nano /etc/proftpd/conf.d/anonymous.conf

Then add the following content to anonymous.conf file and restart proftd service.

<Anonymous ~ftp>
   User                    ftp
   Group                nogroup
   UserAlias         anonymous ftp
   RequireValidShell        off
#   MaxClients                   10
   <Directory *>
     <Limit WRITE>
       DenyAll
     </Limit>
   </Directory>
 </Anonymous>
Configure ProFTPD
Configure ProFTPD

6. Next step is to link apt-mirror path to proftpd path by running a bind mount by issuing the command.

$ sudo mount --bind /opt/apt-mirror/mirror/archive.ubuntu.com/  /srv/ftp/
Mount apt-mirror to ProFTP Path
Mount apt-mirror to ProFTP Path

To verify it run mount command with no parameter or option.

$ mount
Verify Paths
Verify Paths

7. Last step is to make sure that Proftpd server is automatically started after system reboot and mirror-cache directory is also automatically mounted on ftp server path. To automatically enable proftpd run the following command.

$ sudo update-rc.d proftpd enable

To automatically mount apt-mirror cache on proftpd open and edit /etc/rc.local file.

$ sudo nano /etc/rc.local

Add the following line before exit 0 directive. Also use 5 seconds delay before attempting to mount.

sleep 5
sudo mount --bind  /opt/apt-mirror/mirror/archive.ubuntu.com/ /srv/ftp/
Auto Mount Apt Mirrors
Auto Mount Apt Mirrors

If you pull packages from Debian repositories run the following commands and make sure appropriate settings for above rc.local file are enabled.

$ sudo mkdir /srv/ftp/debian
$ sudo mount --bind /opt/apt-mirror/mirror/ftp.us.debian.org/debian/ /srv/ftp/debian/
Debian Repository Setup
Debian Repository Setup

8. For a daily apt-mirror synchronization you can also create a system schedule job to run at 2 AM every day. Run crontab command, select your preferred editor then add the following line syntax.

$ sudo crontab –e
Daily apt-mirror Synchronization
Daily apt-mirror Synchronization

On last line add the following line.

0  2  *  *  *  /usr/bin/apt-mirror >> /opt/apt-mirror/mirror/archive.ubuntu.com/ubuntu/apt-mirror.log
Add Cron Entry for Synchronization
Add Cron Entry for Synchronization

Now every day at 2 AM your system repository cache will synchronize with Ubuntu official mirrors and create a log file.

Step 2: Configure clients

9. To configure local Ubuntu clients, edit /etc/apt/source.list on client computers to point to the IP address or hostname of apt-mirror server – replace http protocol with ftp, then update system.

deb ftp://192.168.1.13/ubuntu trusty universe
deb ftp://192.168.1.13/ubuntu trusty main restricted
deb ftp://192.168.1.13/ubuntu trusty-updates main restricted
## Ad so on….
Configure Clients
Configure Clients

10. To view repositories you can actually open a browser and point to your server IP address of domain name using FTP protocol.

View Local Repositories
View Local Repositories

The same system applies also to Debian clients and servers, the only change needed are debian mirror and sources list.

Also if you install a fresh Ubuntu or Debian system, provide your local mirror manually whit ftp protocol when installer asks which repository to use.

The great thing about having your own local mirror repositories is that you’re always on current and your local clients don’t have to connect to Internet to install updates or software.

Hey TecMint readers,

Exciting news! Every month, our top blog commenters will have the chance to win fantastic rewards, like free Linux eBooks such as RHCE, RHCSA, LFCS, Learn Linux, and Awk, each worth $20!

Learn more about the contest and stand a chance to win by sharing your thoughts below!

Matei Cezar
I'am a computer addicted guy, a fan of open source and linux based system software, have about 4 years experience with Linux distributions desktop, servers and bash scripting.

Each tutorial at TecMint is created by a team of experienced Linux system administrators so that it meets our high-quality standards.

Join the TecMint Weekly Newsletter (More Than 156,129 Linux Enthusiasts Have Subscribed)
Was this article helpful? Please add a comment or buy me a coffee to show your appreciation.

18 Comments

Leave a Reply
  1. Thanks for the info here. I’ve got everything configured but the clients are not upgrading. Running apt-get update on a client shows most packages as get and then ignored.

    Running an apt-get upgrade on the same client results in 0 upgrades, 0 newly installed, 0 to remove, and 0 not upgraded.

    I’ve been contemplating over this a bit, can you suggest what might be the problem?

    Reply
    • @Peter,

      It’s possible that there are no updates available for your system at that time. The system may already have the latest packages installed.

      Sometimes, there might be issues with the repository sources. so make sure that your /etc/apt/sources.list file is correctly configured to point to the appropriate repositories.

      Reply
  2. My issue is a Debian-based machine needed to be reloaded via the internet but can’t because an update (with the idiotic dependencies) is missing from the original OS ISO. The troubling part was the system was loaded (about three weeks ago and working) with the _exact_ same copy of the OS ISO & _exact_ same hardware.

    However, an updated kernel seems to need installing (and complied?) of the internet servers to get the WiFi (Gen. 6) card to work. (A replacement WiFi is not available without spending additional funds. I’ll install MS Windows first.)

    As such, I need a program that will download the entire repository from the internet FTP servers onto a USB pen drive (in my case 270 GB) for off-line installation.

    The configuration of APT-MIRROR is apparently incompetently put together.

    I accidentally discovered that the repository info needed was located in the very same location as the mirror.list config file… /etc/apt/sources.list.

    How do I tell APT-MIRROR to download to: /opt/apt-mirror?

    I find that the needed command lines are missing from this tutorial.

    I have checked all logical places for the downloading files to go (/opt/apt-mirror & /tmp & /home/[USER] & /home/[USER]/Downloads) but they seem just to have vanish.

    Observations and Suggestions:

    Yes, I have programmed and I’m fully aware that scripts are only so flexible but common sense and simplicity seem to be missing from the Unix/Linux communities.

    I would think that a simple FTP [with its directory structure] downloaded for each specific OS (flavor & version) would be far simpler for Administration/Maintenance & End User Support. An option to select a specific storage location (similar to FTP address; e.g.: /media/USBDevice) during OS installation & OS Updates for offline users would be simple enough to accomplish.

    This would easily allow for updating the available online repository, as each program (and versions) would be complete within itself, and allows for use of a specific version of any file achieved with the “automatic” use of the newest version as default.

    This would also allow for any OS to download the data needed for any Unix/Linux version including any ISO images in a single simple program (and likely already available for most OSes today).

    My personal take on the Linux Maintenance systems I’ve seen is that they are overly complex to deal with being from a bygone era of early computing. I fully agree that the computers of the time didn’t have the processing power to deal with the quantities (& varieties) of data currently in use. However, that is not an excuse to stop an update to a simpler system before we are engulfed in the ever engorged leap-frogging that is going on today.

    It would be far better to standardize on a strictly modular c++ standard (complied at installation on each computer) and end the need for the seaming endless dependencies needed today.

    Example: I installed a very basic word processor and it demanded an obsolete “network protocol”?! be installed with it. Well, I dumped that fast.

    Actually, I fully support variety (sorry, I’m bad at spelling) but the endless headaches of dependencies… ARG!

    Thank you for your time.

    Anonymous.

    Reply
  3. Hi All,

    We are using different versions of ubuntu i.e. 14,16,18 & 20. For ubuntu 14 we have desktop versions & others are server versions.

    How we can differentiate between server and desktop OS and how we can set up the local repo to update the different versions mentioned?

    Which Ubuntu version I should choose for the local repo server? Please suggest.

    Thanks.

    Reply
  4. hi Matei Cezar, thanks a lot this great tutorial, i am all done with this process and now my network repository running very well.
    but i want too add some extra package other then default mint and ubuntu repository like google chrome,wpa office, in short i want the repository in that i can add my own deb or tar package and install it in my network. waiting for your great response, thanks in advance.

    Reply
    • I do not recommend adding thirty-party software in apt-mirror list, especially non-free software. Add only repos maintained by the distribution. As far as a know Kingsoft Office doesn’t provide any ppa for Ubuntu, only deb packages. Try to download the latest deb packages for the applications you need and place them in a new directory on the ftp server (/srv/ftp/my_deb for example).

      Reply
  5. I have 2 Debian client and one Debian server (back-up server). The later took less than 20 minutes to pass from Wheezy to Jessie .
    Do you think I would save bandwith if I build a repository basically for the 2 clients,considering when I move from one version to the next I will have to download all the new version packages again.

    Cheers,

    Reply
    • @Luc,
      If you setup a local repository in one of your sever for 2 clients, it will save bandwidth as it downloads and install packages from your local repository..

      In short, your local repository server will first download all packages in your server and then clients will fetch packages from your repository server, this way you can save bandwidth and time..

      Reply
  6. thank you for the information, I’ve tried to configure the server and succeed. but when I try to access via ftp client, it appears username and password. my question is how to fix it ?

    Reply
      • For me also same issue now when I restarted the service I am getting site cant be reached,Please help how to solve this?

        Reply
          • Thanks for replying, now when I restarted the ftp service, service is up but now my ftp://public ip is not loading.Please let me know in aws which ports I need to open.I opened port 20 and 21. But no result.

          • You should open ftp passive incoming ports. Consult ftp conf file fo the passive port range you need to open on aws for your ftp server (ports should be mentined there) or just open all incoming range from 1024 to 65534. Also, check ftp://localhost to test if is working or use a ssh tunnel for ftp service.

  7. I ran apt-mirror and it downloaded a ton of stuff, so far so good, but it only created the ubuntu/pool directory, I don’t have necessary directories such as ubuntu/dists, and this causes the client to fail apt-get update. Any idea what would cause that and how to fix it?

    Reply
  8. Thanks for the great reading material!
    I haven’t had the chance to test your setup in production but I do have one set up with nginx as a web server repository. It used by just about 1000 clients and it is really fast.
    The hardware has more of a 2003-2004 feeling about it but it does the job flawlessly.

    Reply

Got Something to Say? Join the Discussion...

Thank you for taking the time to share your thoughts with us. We appreciate your decision to leave a comment and value your contribution to the discussion. It's important to note that we moderate all comments in accordance with our comment policy to ensure a respectful and constructive conversation.

Rest assured that your email address will remain private and will not be published or shared with anyone. We prioritize the privacy and security of our users.