Setup Web Server (apache) and DNS (bind) on a MacMini using Homebrew

Contents

  1. XCODE and BBEdit command line tools
  2. BREW
    1. Setup:
    2. Maintenance:
    3. Debugging:
  3. APACHE
    1. Setup:
    2. Maintenance:
    3. Debugging:
  4. HTTPD config edits HTTP2, headers, userdirs, proxy
    1. HTTPD config basic edits:
    2. HTTPD config edits to enable http2
    3. Enable full filename display in case of directory index:
    4. Enable example.net/~user.
    5. Headers for security hardening
    6. Enable (reverse) proxies.
  5. HTTPD config edits for vhosts
    1. HTTPD config edits enable vhosts:
  6. Let's Encrypt - certbot
  7. HTML default header
  8. DNS - bind and rndc
    1. Setup:
    2. Maintenance:
    3. Debugging:
    4. Zone file specials MX, DMARC, SPF, CAA:
  9. DNSSEC
    1. Setup:
    2. Debugging:
  10. DANE
    1. Setup:
  11. DKIM
    1. Setup:
    2. Debugging:
  12. Postfix and Dovecot
    1. Here some options to get a higher internet.nl score:
  13. SSH on a different port number
  14. SSH tunnel for SMB for timemachine
    1. Setup
    2. Maintenance:
    3. Debugging:
  15. TimeMachine
    1. Setup:
    2. Debugging:
  16. How to solve calendar problems
  17. Network Performance tools
  18. Common pitfalls, problems, useful commands
  19. MacMini Setup (for SC/webcam/demo)
  20. Online references
  21. How this page is made

XCODE and BBEdit command line tools

Apple's command line tools need to be installed, XCode is not needed.
  • xcode-select --install

If needed to re-install command tools do this command first:

  • sudo rm -rf /Library/Developer/CommandLineTools

Install BBEdit from https://www.barebones.com/products/bbedit/index.html

Start BBEdit and set up its command line tools under the BBEdit menu.

BREW

Setup:

  • ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  • brew install openldap libiconv
For more information see: https://docs.brew.sh/FAQ

Maintenance:

To upgrade all installed programs to newest version:
  • brew update
  • brew upgrade
  • brew cleanup
  • brew services list
  • sudo brew services restart --all

Debugging:

Check version and installation:
  • brew doctor
  • brew --version
If needed to correct permissions:
  • sudo chown -R "$USER":admin /usr/local
  • sudo chown -R "$USER":admin /Library/Caches/Homebrew
If brew directories are not in the path:
  • echo 'export PATH="/usr/local/sbin /usr/local/bin $PATH"' >> ~/.zshrc

APACHE

Setup:

  • brew install httpd
  • sudo brew services start httpd

Maintenance:

To restart httpd:

  • sudo brew services restart httpd
  • sudo apachectl stop
  • sudo apachectl -k restart

Debugging:

In a separate windo show dynamically the tail of the error and access logfile:
  • tail -n 200 -f /usr/local/var/log/httpd/error_log
  • tail -n 200 -f /usr/local/var/log/httpd/access_log
If install has problems because of previous versions:
  • sudo launchctl unload -w /System/Library/LaunchDaemons/org.apache.httpd.plist
  • sudo apachectl stop
  • sudo brew services restart httpd

Check if deamon is running:

  • ps -aef | grep httpd
To get setup paths:
  • sudo apachectl -S
Test conf files, must be with sudo to test certificates:
  • sudo apachectl configtest

HTTPD config edits HTTP2, headers, userdirs, proxy

HTTPD config basic edits:

  • bbedit /usr/local/etc/httpd/httpd.conf
  1. Listen 8080 => Listen 80
  2. enable ==> LoadModule deflate_module lib/httpd/modules/mod_deflate.so
  3. enable ==> LoadModule rewrite_module lib/httpd/modules/mod_rewrite.so
  4. ServerAdmin admin@example.net
  5. #ServerName www.example.net:8080 ==> ServerName localhost:80
  6. enable ==> Include /usr/local/etc/httpd/extra/httpd-autoindex.conf

HTTPD config edits to enable http2

  • bbedit /usr/local/etc/httpd/httpd.conf
  1. disable ===> #LoadModule mpm_prefork_module lib/httpd/modules/mod_mpm_prefork.so
  2. enable ===> LoadModule mpm_event_module lib/httpd/modules/mod_mpm_event.so
  3. enable ===> LoadModule http2_module lib/httpd/modules/mod_http2.so
  4. add ===> Protocols h2 h2c http/1.1

Enable full filename display in case of directory index:

  • bbedit /usr/local/etc/httpd/extra/httpd-autoindex.conf

add NameWidth=* to the line IndexOptions FancyIndexing HTMLTable VersionSort

  1. ===> IndexOptions FancyIndexing HTMLTable VersionSort NameWidth=*

Enable example.net/~user.

In httpd.conf:
  • bbedit /usr/local/etc/httpd/httpd.conf
  1. enable ==> LoadModule userdir_module lib/httpd/modules/mod_userdir.so
  2. enable ==> Include /usr/local/etc/httpd/extra/httpd-userdir.conf
Then:
  • bbedit /usr/local/etc/httpd/extra/httpd-userdir.conf
  1. Change public_html into Sites

Headers for security hardening

See:
In httpd.conf:
  • bbedit /usr/local/etc/httpd/httpd.conf
  1. enable ==> LoadModule headers_module lib/httpd/modules/mod_headers.so
  2. add at the end of the conf file:

    <IfModule headers_module>
    #  header set Strict-Transport-Security "max-age=15768000; includeSubDomains"
      header set Strict-Transport-Security "max-age=15768000"
      header set X-XSS-Protection "1; mode=block"
      header set X-Frame-Options "SAMEORIGIN"
      header set X-Content-Type-Options "nosniff"
      header set Referrer-Policy "same-origin"
    #  header set Content-Security-Policy: "default-src 'self' '*' ;"
    </IfModule>

Enable (reverse) proxies.

See:

In httpd.conf:

  • bbedit /usr/local/etc/httpd/httpd.conf
  1. enable ==> #LoadModule proxy_module lib/httpd/modules/mod_proxy.so
  2. LoadModule proxy_http_module lib/httpd/modules/mod_proxy_http.so
Then for port based:
  • bbedit /usr/local/etc/httpd/extra/httpd-vhosts.conf
  • Listen 12345
    <VirtualHost *:12345>
        ProxyPass "/" "http://127.0.0.1:12344/"
        ProxyPassReverse "/" "http://127.0.0.1:12344/"
    </VirtualHost>

  • Similar can be done in the ssl vhost files below to embed traffic in ssl.

    HTTPD config edits for vhosts

    HTTPD config edits enable vhosts:

    • bbedit /usr/local/etc/httpd/httpd.conf
    1. enable ==> LoadModule vhost_alias_module lib/httpd/modules/mod_vhost_alias.so
    2. enable ==> Include /usr/local/etc/httpd/extra/httpd-vhosts.conf
    and for vhosts including a catch directory. The catch directory is the default webroot where all requests go to if there is not a specific webroot for that (sub)domain defined. Therefore, it must be the first in the list. Here an example for example.net edit:
    • bbedit /usr/local/etc/httpd/extra/httpd-vhosts.conf
    <VirtualHost *:80>
        DocumentRoot "/usr/local/var/www"
        ServerName catch.example.net
    </VirtualHost>
    <Directory "/usr/local/var/www">
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>

    <VirtualHost *:80>
        DocumentRoot "/Users/example/Sites"
        ServerName example.net
        ServerAlias www.example.net
    </VirtualHost>
    <Directory "/Users/example/Sites">
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>

    Let's Encrypt - certbot

    • sudo install -d -o $(whoami) -g admin /usr/local/Frameworks
    • brew install certbot
    HTTPD edits:
    • bbedit /usr/local/etc/httpd/httpd.conf

    Uncomment:

    LoadModule ssl_module modules/mod_ssl.so
    LoadModule socache_shmcb_module modules/mod_socache_shmcb.so
    Include /usr/local/etc/httpd/extra/httpd-ssl.conf

    add at the end:

    Include /usr/local/etc/httpd/extra/httpd-vhosts-le-ssl.conf

    For best security edit in httpd-ssl.conf:
    • bbedit /usr/local/etc/httpd/extra/httpd-ssl.conf
    1. Make sure these parameters have following values:
      • SSLProtocol             all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
      • SSLCipherSuite "ECDHE-ECDSA-AES256-GCM-SHA384 TLS_AES_256_GCM_SHA384 ECDHE-ECDSA-CHACHA20-POLY1305 TLS_CHACHA20_POLY1305_SHA256 ECDHE-ECDSA-AES128-GCM-SHA256 TLS_AES_128_GCM_SHA256 ECDHE-RSA-AES256-GCM-SHA384 TLS_AES_256_GCM_SHA384 ECDHE-RSA-CHACHA20-POLY1305 TLS_CHACHA20_POLY1305_SHA256 ECDHE-RSA-AES128-GCM-SHA256 TLS_AES_128_GCM_SHA256 ECDHE-ECDSA-AES256-SHA384 ECDHE-ECDSA-AES256-SHA ECDHE-ECDSA-AES128-SHA256 ECDHE-ECDSA-AES128-SHA  ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES256-SHA ECDHE-RSA-AES128-SHA256 ECDHE-RSA-AES128-SHA "
      • SSLProxyProtocol             all -SSLv2 -SSLv3 -TLSv1 -TLSv1.1
      • SSLProxyCipherSuite "ECDHE-ECDSA-AES256-GCM-SHA384 TLS_AES_256_GCM_SHA384 ECDHE-ECDSA-CHACHA20-POLY1305 TLS_CHACHA20_POLY1305_SHA256 ECDHE-ECDSA-AES128-GCM-SHA256 TLS_AES_128_GCM_SHA256 ECDHE-RSA-AES256-GCM-SHA384 TLS_AES_256_GCM_SHA384 ECDHE-RSA-CHACHA20-POLY1305 TLS_CHACHA20_POLY1305_SHA256 ECDHE-RSA-AES128-GCM-SHA256 TLS_AES_128_GCM_SHA256 ECDHE-ECDSA-AES256-SHA384 ECDHE-ECDSA-AES256-SHA ECDHE-ECDSA-AES128-SHA256 ECDHE-ECDSA-AES128-SHA  ECDHE-RSA-AES256-SHA384 ECDHE-RSA-AES256-SHA ECDHE-RSA-AES128-SHA256 ECDHE-RSA-AES128-SHA "
      • SSLHonorCipherOrder     on
      • SSLSessionTickets       off
      • SSLCompression          off
      • SSLOptions +StrictRequire
      • # Add vhost name to log entries:
      • LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" vhost_combined
      • LogFormat "%v %h %l %u %t \"%r\" %>s %b" vhost_common
    2. comment out the certs
    3. change listen 8443 ==> listen 443
    4. comment out everything in the virtual host block including #<VirtualHost _default_:8443> and #</VirtualHost>
    The vhost and vhost-le-ssl edits:

    The idea is that all requests that get through on port 80 or with incorrect url's/domain names or with http://catch.example.net end up in:
    • "/usr/local/var/www"
    and all requests that come in via https and correct domain names go to the correct webroots of those domains.

    For that purpose a rewrite rule takes care of redirection. This is in httpd-vhosts.conf
    I use the following domain construction for example.net:
    • catch example.net
      • catches all (sub)domains for which no other webroot is defined is
    • example.net
      • this goes to the normal webroot of the domain example.net
    • www.example.net
      • also goes to the normal webroot of the domain example.net
    • sub1.example.net
      • a subdomain of example.net with its own webroot
    in httpd-vhosts.conf:
    • bbedit /usr/local/etc/httpd/extra/httpd-vhosts.conf
    <VirtualHost *:80>
        DocumentRoot "/usr/local/var/www"
        ServerName catch.example.net
    </VirtualHost>
    <Directory "/usr/local/var/www">
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>

    <VirtualHost *:80>
        DocumentRoot "/usr/local/var/www"
        ServerName example.net
        ServerAlias www.example.net
        ServerAlias sub1.example.net
    RewriteEngine on
    RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
    </VirtualHost>

    Note that the Directory directive for example and sub1 is moved to the httpd-vhosts-le-ssl.conf file. This ensures that if that file does not get processed, those webroots are not exposed.

    and create/edit httpd-vhosts-le-ssl.conf:
    • bbedit /usr/local/etc/httpd/extra/httpd-vhosts-le-ssl.conf
    <VirtualHost *:443>
        DocumentRoot "/usr/local/var/www"
        ServerName catch.example.net
    Include  /usr/local/etc/httpd/extra/httpd-vhosts-le-ssl-options.conf
    </VirtualHost>

    <VirtualHost *:443>
        DocumentRoot "/Users/example/Sites"
        ServerName example.net
        ServerAlias www.example.net
    Include  /usr/local/etc/httpd/extra/httpd-vhosts-le-ssl-options.conf
    </VirtualHost>
    <Directory "/Users/example/Sites">
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>
    <VirtualHost *:443>
        DocumentRoot "/Users/sub1example/Sites"
        ServerName sub1.example.net
    Include  /usr/local/etc/httpd/extra/httpd-vhosts-le-ssl-options.conf
    </VirtualHost>
    <Directory "/Users/sub1example/Sites">
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
    </Directory>

    And create the file:
    • bbedit /usr/local/etc/httpd/extra/httpd-vhosts-le-ssl-options.conf
    with content (correct name of the keys!):

    #all necessary settings in this file comes from /etc/letsencrypt/options-ssl-apache.conf
    # therefore that file is not needed anymore. This one needs to be in every ssl enabled vhost.

    SSLEngine on


    SSLCertificateFile /etc/letsencrypt/live/example.net/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/example.net/privkey.pem

    To request the certificates:
    • sudo certbot --apache
    or if one first wants to do a number of test runs for debugging:
    • sudo certbot --apache --staging
    for after successful testing forcing a full new certificate:
    • sudo certbot --apache --force-renewal
    for production:
    • sudo certbot renew

    For renewal we have to make a launchd plist, see:

    However, the plist in the above solutions don't not work because somehow the PATH variable of the running deamon is not correct. Therefore, we add an environment variable in the plist that sets the path.

    Create plist:

    cat >com.letsencrypt.renew.plist <<EOF
    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
        <dict>
            <key>Label</key>
            <string>com.letsencrypt.renew</string>
            <key>RunAtLoad</key>
            <true/>
            <key>EnvironmentVariables</key>
            <dict>
                 <key>PATH</key>
                  <string>/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin</string>
            </dict>
            <key>ProgramArguments</key>
            <array>
                <string>/usr/local/bin/certbot</string>
                <string>renew</string>
            </array>
            <key>StartCalendarInterval</key>
            <dict>
                <key>Hour</key>
                <integer>4</integer>
                <key>Minute</key>
                <integer>56</integer>
            </dict>
            <key>StandardErrorPath</key>
              <string>/tmp/local.certbot.renew.log</string>
            <key>StandardOutPath</key>
            <string>/tmp/local.certbot.renew.log</string>
        </dict>
    </plist>
    EOF
    • sudo mv com.letsencrypt.renew.plist /Library/LaunchDaemons/com.letsencrypt.renew.plist
    • sudo chmod 644 /Library/LaunchDaemons/com.letsencrypt.renew.plist
    • sudo chown root:admin /Library/LaunchDaemons/com.letsencrypt.renew.plist
    • sudo launchctl load -w /Library/LaunchDaemons/com.letsencrypt.renew.plist
    • sudo launchctl list | grep -i letsencrypt

    If test runs are needed:

    • sudo certbot renew --dry-run

    If pre, post or deploy scripts are needed, those go here:

    • /private/etc/letsencrypt/renewal-hooks
    Debugging:
    • tail -f -n 40 /tmp/local.certbot.renew.log
    • sudo tail -f -n 400 /var/log/letsencrypt/letsencrypt.log

    To list the certificates:

    • sudo certbot certificates

    Some more debugging:

    The configuration file is at: /etc/letsencrypt/renewal/

    HTML default header

    A good web pages header for .html files:

    <!DOCTYPE html>
    <html>
      <head>
        <title>title</title>
        <meta charset="UTF-8">
        <meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=0.25, maximum-scale=4.0, user-scalable=yes">
        <meta name="description" content="title">
        <meta name="author" content="Cees de Laat">
      </head>
      <body style="font-family: Helvetica,Arial,sans-serif;" text="#ffffff" bgcolor="#000000" link="#ffcc66" alink="#ff9900" vlink="#ffff66">

    DNS - bind and rndc

    Setup:

    • brew install bind
    • rndc-confgen -a

    Edit the conf file:

    • bbedit /usr/local/etc/named.conf
    Add in the Options section the directory location of zone files, hiding version for security and rate limiting to prevent ddos amplification:

    directory "/usr/local/var/named";
    version "unknown";
    rate-limit {
               responses-per-second 15;
               window 5;
               log-only no;
    };

    And create the zone files in:

    • /usr/local/var/named/
    Start bind:
    • sudo brew services start bind

    Maintenance:

    • rndc sync -clean
    • rndc reload
    • sudo brew services restart bind
    • sudo brew services stop bind

    Debugging:

    First set up a separate window with tail of logging:

    • tail -f -n 40 /usr/local/var/log/named/named.log

    Some checks:

    • rndc status
    • named-checkconf -z /usr/local/etc/named.conf
    • host -t ns example.net

    Zone file specials MX, DMARC, SPF, CAA:

    $TTL 3600
    example.net.                  IN SOA    ns1.example.net. sysadmin.example.net. (
                                                           2019090801
                                                           21600
                                                           900
                                                          1209600
                                                          3600)
                                          IN NS       ns1.example.net.
                                          IN NS       ns2.example.net.
                                          IN NS       ns3.example.net.
                                          IN A         111.111.111.111
                                          IN AAAA  1111:2222:3333:4444:5555:6666:7777:8888
                                          IN CAA     128 issue "letsencrypt.org"
                                          IN MX      10 smtp.example.net.
                                          IN TXT     "v=spf1 a mx -all"
    _dmarc.example.net.    IN TXT     "v=DMARC1; p=none; pct=100; aspf=r; rua=mailto:sysadmin@example.net"

    DNSSEC

    Setup:

    Based on: The structure used for naming the zone files is:
    • The master zone files: edit those to change records, etc.
      • /usr/local/var/named/example.net.zone
    • The keys:
      • /usr/local/var/named/keys/
    • Bind can automatically sign and will produce intermediate files in the same directory as the zone files.
    • mkdir /usr/local/var/named/keys

    Then we need to once generate the keys for each zone file on the master DNS server. That can be done with the following commands:

    • dnssec-keygen -3 -a ECDSAP256SHA256 -K /usr/local/var/named/keys/ example.net
    • dnssec-keygen -3 -a ECDSAP256SHA256 -f KSK -K /usr/local/var/named/keys/ example.net
      For a convenient script that does this:
    • bbedit /usr/local/sbin/keygen.sh
    and fill it with:

    #!/bin/sh
    KEYDIR="/usr/local/var/named/keys/"
    /usr/local/sbin/dnssec-keygen -3 -a ECDSAP256SHA256 -K $KEYDIR  $1
    /usr/local/sbin/dnssec-keygen -3 -a ECDSAP256SHA256 -f KSK -K $KEYDIR $1

    Make it executable:
    • chmod +x /usr/local/sbin/keygen.sh

    Enable DNSSEC in named.conf by adding the following configuration directives inside options{ }:

    • bbedit  /usr/local/etc/named.conf
    in the main options add:

    key-directory "/usr/local/var/named/keys";

    in each of the to be signed signed zones add:

        auto-dnssec maintain;
        inline-signing yes;

    Restart bind to sign the zones and load them in master and slaves:
    • sudo brew services restart bind

    Now we need the Delegation Signing records to supply to the parent zone manager. Easiest is using dig and dnssec-dsfromkey:

    • dig @127.0.0.1 dnskey example.net | dnssec-dsfromkey -1 -2 -f - example.net

    Debugging:

    Check if for the DNSKEY record using dig on the same server.
    • dig DNSKEY example.net. @localhost +multiline
    Check for the presence of RRSIG records.
    • dig A example.net. @localhost +noadditional +dnssec +multiline
    Check DS records:
    • dig +trace +noadditional DS example.net. @8.8.8.8 | grep DS
    Check DNSSEC:
    • http://dnssec-debugger.verisignlabs.com
    • http://dnsviz.net/

    DANE

    Setup:

    Work in progress. Let's Encrypt makes that it needs to be renewed every 3 months, or needs an adapted server certificate.

    Based on: Make TLSA records:

    Download eerst de certificaten van de server (server.pem) en de CA (CA.pem) naar uw werkstation. In de voorbeelden nemen we aan dat uw mailserver de FQDN mail.example.nl heeft.
    Aanmaken van een '2 1 1'-record:
    $ python tlsa --create --port 25 --usage 2 --selector 1 --certificate CA.pem mail.example.nl
    Aanmaken van een '3 1 1'-record:
    $ python tlsa --create –-port 25 –-usage 3 --selector 1 --certificate server.pem mail.example.nl

    possible rndc commands:
    • rndc reload
    • rndc reconfig

    DKIM

    Setup:

    To Be Done

    Based on:

    Debugging:

    Check for the DKIM record using dig on the same server.
    • dig

    Postfix and Dovecot

    Here some options to get a higher internet.nl score:

    Based on:
    smtp_tls_security_level = encrypt
    smtpd_tls_security_level = encrypt

    smtpd_tls_protocols = TLSv1.3, TLSv1.2, !TLSv1.1, !TLSv1, !SSLv2, !SSLv3
    smtp_tls_protocols = TLSv1.3, TLSv1.2, !TLSv1.1, !TLSv1, !SSLv2, !SSLv3
    smtp_tls_ciphers = medium
    smtpd_tls_ciphers = medium
    smtpd_tls_mandatory_protocols = TLSv1.3, TLSv1.2, !TLSv1.1, !TLSv1, !SSLv2, !SSLv3
    smtp_tls_mandatory_protocols = TLSv1.3, TLSv1.2, !TLSv1.1, !TLSv1, !SSLv2, !SSLv3
    smtp_tls_mandatory_ciphers = high
    smtpd_tls_mandatory_ciphers = high

    smtpd_tls_mandatory_exclude_ciphers = MD5, DES, ADH, RC4, PSD, SRP, 3DES, eNULL, aNULL
    smtpd_tls_exclude_ciphers = MD5, DES, ADH, RC4, PSD, SRP, 3DES, eNULL, aNULL
    smtp_tls_mandatory_exclude_ciphers = MD5, DES, ADH, RC4, PSD, SRP, 3DES, eNULL, aNULL
    smtp_tls_exclude_ciphers = MD5, DES, ADH, RC4, PSD, SRP, 3DES, eNULL, aNULL
    tls_preempt_cipherlist = yes
    tls_ssl_options = NO_RENEGOTIATION

    smtpd_tls_dh2048_param_file = <PATH TO>/dh2048.pem
    smtpd_tls_dh1024_param_file = <PATH TO>/dh1024.pem
    smtpd_tls_dh512_param_file = <PATH TO>/dh512.pem

    SSH on a different port number

    This procedure and port numbers come from:

    See also:

    Procedure:
    • sudo vi /etc/services
      • Change the port number in:
      • ssh 22/udp # SSH Remote Login Protocol
      • ssh 22/tcp # SSH Remote Login Protocol
    • sudo vi /etc/ssh/ssh_config
      • uncomment the following: Port 22
        and change that port number in the desired one.
    • Restart the ssh daemon.
    • sudo launchctl unload -w /System/Library/LaunchDaemons/ssh.plist
    • sudo launchctl load -w /System/Library/LaunchDaemons/ssh.plist

    SSH tunnel for e.g. VNC. Here we assume ssh over port 12345. After this tunnel setup one can ust vnc to 127.0.0.1:5901 to reach the other machine:

    • ssh -p 12345 -v -L 5901:127.0.0.1:5900 destination

    SSH tunnel for SMB for timemachine

    Setup

    Purpose is to tunnel port 445 (SMB) through a ssh tunnel to get past port filtering of some ISP's to allow Apple Timemachine to work to remote servers. This is implemented by creating a network alias to loopback interface in order to have a local tunnel endpoint on port 445 (SMB). Then create keys for ssh if not already there, copy the key(s) to the timemachine server for password-less login. Then create a ssh tunnel from the local alias address to the remote server. The remote disk for backup can then be mounted as a normal smb disk by addressing the local alias address. The procedure:

    Create an alias to be able to bind port 445 locally to a tunnel:
    • sudo ifconfig lo0 alias 127.0.0.2/32
    If not done before: create ssh key to authenticate to remote host and move public key to remote server:
    • ssh-keygen
    • ssh-copy-id remoteuser@example.net -p <port>
    Test the ssh for both user and root. Need to test root since we want to forward the privileged port 445 later. This also ensures that the server is in the list of known hosts.
    • ssh -p <port> remoteuser@example.net
    • sudo ssh -p <port> -i /Users/localuser/.ssh/id_rsa user@example.net
    Then set up the tunnel in a way that (hopefully) keeps the tunnel open:
    • sudo ssh -f -C -N -p <port> -i /Users/localuser/.ssh/id_rsa -L 127.0.0.2:445:127.0.0.1:445 -o ServerAliveCountMax=3 -o ServerAliveInterval=15 -o ExitOnForwardFailure=yes remoteuser@example.net
    One can also try to use autossh for more tunnel robustness. If not already installed, install autossh and ssh to have the matching version:
    • brew install ssh
    • brew install autossh

    Start the tunnel:

    • sudo autossh -M 0 -f -C -N -p <port> -i /Users/localuser/.ssh/id_rsa -L 127.0.0.2:445:127.0.0.1:445 -o ServerAliveCountMax=3 -o ServerAliveInterval=15 -o ExitOnForwardFailure=yes remoteuser@example.net
    To create the loopback alias address and the autossh each time the machine is booted, create a startup.sh script at /usr/local/var and Launch Daemon property list file located at /Library/LaunchDaemons/localhost.startup.plist.

    Copy next two pieces of text into your terminal to create the files:

    cat > /usr/local/var/startup.sh <<EOF
    #!/bin/zsh
    ifconfig lo0 alias 127.0.0.2/32
    autossh -M 0 -f -C -N -p <port> -i /Users/localuser/.ssh/id_rsa -L 127.0.0.2:445:127.0.0.1:445 -o ServerAliveCountMax=3 -o ServerAliveInterval=15 -o ExitOnForwardFailure=yes remoteuser@example.net
    EOF


    cat >/usr/local/var/localhost.startup.plist<<EOF
    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
        <dict>
            <key>Label</key>
            <string>localhost.startup</string>
            <key>RunAtLoad</key>
            <true/>
            <key>EnvironmentVariables</key>
            <dict>
                 <key>PATH</key>
                  <string>/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin</string>
            </dict>
            <key>ProgramArguments</key>
            <array>
                <string>/usr/local/var/startup.sh</string>
            </array>
        </dict>
    </plist>
    EOF

    Then set attributes of the files and copy the plist to the correct place
    • chmod +x /usr/local/var/startup.sh
    • sudo chmod 644 /usr/local/var/localhost.startup.plist
    • sudo mv /usr/local/var/localhost.startup.plist /Library/LaunchDaemons/localhost.startup.plist
    • sudo chown root:admin /Library/LaunchDaemons/localhost.startup.plist
    • sudo launchctl load -w /Library/LaunchDaemons/localhost.startup.plist
    • sudo launchctl list | grep -i startup
    • ifconfig | grep 127
    • ps aux | grep ssh

    Maintenance:

    NOTE: this all works but is not persistent. After a reboot the following needs to be redone:
    • sudo ifconfig lo0 alias 127.0.0.2/32
    • sudo ssh -f -C -N -p <port> -i /Users/localuser/.ssh/id_rsa -L 127.0.0.2:445:127.0.0.1:445 -o ServerAliveCountMax=3 -o ServerAliveInterval=15 -o ExitOnForwardFailure=yes remoteuser@example.net

    Debugging:

    Check if alias is up:
    • ifconfig | grep 127
    Check if tunnel is present:
    • ps aux | grep tunnel

    TimeMachine

    Setup:

    Here are TimeMachine cli commands to start up, stop, get info, etc. The info comes from:
    The basics of the tmutil command can be found by typing
    • man tmutil
    The man page tells you what you can do with this command. For example, to turn Time Machine on or off, you can run these commands:
    • sudo tmutil enable
    • sudo tmutil disable

    To get the status of anything going on or progressing:

    • tmutil status
    If you want to run a Time Machine backup right away, on a Mac that either has Time Machine disabled, or, say, just before updating to a new version of OS X, you can run this command:
    • tmutil startbackup
    This is the same as choosing Back Up Now from the Time Machine menu in the menu bar at the top of your screen. And if you ever want to stop a backup, just run this:
    • tmutil stopbackup
    Save disk space on your laptop Since your laptop isn’t always connected to its backup disk, Time Machine retains “local snapshots,” or files that it will copy to your backup disk the next time it is available. However, these local snapshots take up space, and you may want to turn this feature off if you don’t have much room on your hard disk. You can turn off (Running this command will also delete any local snapshots.) and on with the following commands:
    • sudo tmutil disablelocal
    • sudo tmutil enablelocal
    You can exclude certain files and folders from your Time Machine backups from the Time Machine pane in System Preferences. Naturally, you can also do this from the command line, too. Run this command:
    • sudo tmutil addexclusion <path of folder/file to exclude>
    The tmutil addexclusion command has an interesting property: it's sticky. When you use this command, the item you exclude remains in the Time Machine exclusion list even if you move it, which is not the case when you exclude items from the Time Machine preference pane. If you use the above command with the -p flag, then it will not be sticky, and will be the same as an exclusion you add from the Time Machine preference pane.

    If you’re managing a remote Mac, such as a server, you may want to change Time Machine settings for that computer. You can start by finding where Time Machine backups are stored. Run this command:
    • tmutil destinationinfo
    To change the destination, you can use two commands. First, remove the current destination like this:
    • tmutil removedestination ......
    In place of ..... , type in the text string returned by the destinationinfo command. Then run this command to set up a new destination disk:
    • tmutil setdestination volume_name
    Replace volume_name with the name of the disk or volume you want to use. You can add multiple destinations as well, since Time Machine can rotate its backups on several disks or volumes. See man tmutil for more on setting up multiple backup destinations. (You can now do this without the command line too, see “How to create redundant Time Machine backups.”)

    Time Machine saves a lot of backups: one per hour for the past 24 hours; one a day for the past week; and one a week before that. You can get a list of all the backups on your Time Machine disk with this command:
    • tmutil listbackups
    The tmutil command offers many other options, such as the ability to inherit destinations, perform detailed comparisons of backups, restore items and much more. See man tmutil to find out all that you can do.

    Debugging:

    This procedure is to correct errors like: “Time Machine completed a verification of your backups. To improve reliability, Time Machine must create a new backup for you.”. It may or may not work. The procedures come from:
    The steps:
    • sudo chflags -R nouchg   <the backup bundle>
    • sudo hdiutil attach -nomount -noverify -noautofsck <the backup bundle>
    • sudo tail -f /var/log/fsck_hfs.log
    • sudo fsck_hfs -drfy -c 2g /dev/diskXs2
    If you get a message in the fsck_hfs.log along the lines of " RebuildBTree – record x in node y is not r" then try:
    • fsck_hfs -p /dev/diskXs2
    • fsck_hfs -drfy -c 2g /dev/diskXs2

    To scan for bad blocks:

    • /sbin/fsck_hfs -S /dev/diskXs2
    When succeeded:
    • hdiutil detach /dev/diskXs2
    When complete, you need to edit an plist file within the sparsebundle that records the state of the backup. On the top level of the sparsebundle find a file called "com.apple.TimeMachine.MachineID.plist".
    • bbedit <the backup bundle>/com.apple.TimeMachine.MachineID.plist
    Remove these two nodes:

    <key>RecoveryBackupDeclinedDate</key>

    <date>{whatever-the-date}</date>

    Finally you want to change:

    <key>VerificationState</key>
<integer>2</integer>
    to
    <key>VerificationState</key>
<integer>0</integer>

    Now Time Machine can give it another go. After the (long) verification step, backups should proceed once again.

    To find log file entries:
    • log show --predicate 'subsystem == "com.apple.TimeMachine"' --info | grep 'upd: (' | cut -c 1-19,140-999

    How to solve calendar problems

    How to solve the error: "Apple Calendar Can’t Save Event to Exchange":

    See: https://michaelkummer.com/tech/apple/macos-calendar-exchange/

    • Quit Calendar application (Command + Q)
    • Quit Apple Mail
    • Open Activity Monitor (through Spotlight or Launchpad)
    • Search for “Calendar” and quit all the relevant processes
    • Open Finder and navigate to ~/Library/Calendars
    • Double-check Activity Monitor to make sure no Calendar-related processes are running
    • Delete cache files
    • Relaunch Calendar

    Network Performance tools

    See:
    brew install iperf iperf3 nuttcp bwctl owamp
    iperf
    Server:
    • iperf -s -i 4 -w 5m
    Client:
    • iperf -i 4 -t 1000 -N -w 5M -l 1M -c [servername]
    iperf3
    Server:
    • iperf3 -s -i 5
    Client:
    • iperf3 -i 4 -t 1000 -N -w 5M -l 1M -c [servername]
    nuttcp
    On the server:
    • nuttcp -S
    Client:
    • nuttcp [servername]
    This runs a 10 second test, only on ipv4

    Common pitfalls, problems, useful commands

    • After a brew upgrade some services may not work. One difficult to diagnose problem may be the firewall settings in MacOSX. It blocks by default incoming connections on unsigned bin's and executables need to be added in the list in the System Preferences Firewall settings. Also note that just putting there the /usr/local/bin/executable will not work since brew puts aliasses (links) there. Follow the link to the real bin. Same for /usr/local/sbin .
    • After a "brew upgrade" the output may display an error that an old directory could not be removed and a sudo command is displayed. In reality the "brew upgrade" did not complete, so execute that sudo command and repeat "brew upgrade" and displayed error - sudo commands until nothing is being done anymore by that command.
    • Somehow launchd has a different or incomplete PATH environment. Caused certbot renew to initially fail.
    • if you need to log out another user:
      • ps awwwwux | grep loginwindow
      • kill the pid of that user with:
      • sudo kill -9 [pid]
    • Support files can be found HERE.
    • SSH plus Screensharing
      • ssh -p 22 -N -L 5999:localhost:5900 user@example.net
      • vnc://localhost:5999
    • Starting/stopping remote desktop. The commands in this article https://support.apple.com/en-us/HT201710  work with Apple Remote Desktop 3.2 and later. Here are commands that you can use:
      • Restart the ARD Agent and helper:
        • sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -restart -agent
      • Turn on Remote Desktop Sharing, allow access for all users, and enable the menu extra:
        • sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -activate -configure -allowAccessFor -allUsers -privs -all -clientopts -setmenuextra -menuextra yes
      • Turn on Remote Desktop Sharing, allow access for specified users:
        • sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -activate -configure -allowAccessFor -specifiedUsers
      • You must use the -configure, -access, and -privs options in a separate command to specify the set of users and their access privileges. For example, this command is for users with the short names "teacher" and “student." It gives them access to observe (but not control) the computer, and to send text messages:
        • sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -configure -users teacher,student -access -on -privs -ControlObserve -ObserveOnly -TextMessages
      • Unlike other kickstart options, you can’t combine the allowAccessFor options with other kickstart options. You must use it as in the last two samples above. You might have to call kickstart more than once to finish a computer’s setup. Remove access privileges for specified users ("student" in this example):
        • sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -configure -users student -access -off
      • Disable ARD Agent and remove access privileges for all users:
        • sudo /System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart -deactivate -configure -access -off
    • Shell need to change from bash to zsh now:
      • The default interactive shell is now zsh. To update your account to use zsh, please run `chsh -s /bin/zsh`. For more details, please visit https://support.apple.com/kb/HT208050.
        • cp .bash_profile .zshrc
        • chsh -s /bin/zsh
      • restart terminal
        • brew doctor

    MacMini Setup (for SC/webcam/demo)

    This section describes how to set up mac-minies for use as webcam source. Note that EVOCAM is now unsupported and there are probably better ways to achieve webcam streaming. Also the privacy laws allow less and less.

    Note that some of the software can also be found here: files directory
    • Users & Groups
      • SNE-Admin
        • sne-admin
      • SNE-demo
        • sne-demo
      • Login Options
        • auto login sne-demo
    • Power settings
      • never sleep computer
      • restart after power fail
      • prevent display sleep
      • awake with net access
      • start up 8h00 in the morning
    • Desktop & Screen Saver
      • no screen saver
    • Sharing
      • Screen Sharing
      • File Sharing
      • Remote login
    • Security
      • turn off screen lock
      • enable location services
      • no filevault
    • Date & Time
      • automatic time adjustment
    • Display Menu.app
    • MenuMeters
    • BBEdit
    • Deskpicture SNE logo
    • Team Viewer Setup
    • To keep mac from sleeping unexpectedlythat stops video
      • in terminal: caffeinate -di &
    • EvoCam Setup
      • Evocam5 download
        • serial ES56-MUDX-9LD6-BRAG
        • Note:
          • EvoCam 4 crashes now and then but does recording fine, however is 32 bit app!
          • EvoCam 5 is more stable but gives unusable recordings, is 64 bit!
          • NOTE: the low resolution serve must start first, then the high resolution, otherwise video is low quality.
      • Settings
        • Preferences
          • web server port nr 10456
          • Log Web Server access
          • auto-open docs from previous session at startup
          • make sure the low res is loaded first.
          • Finder Cam1.settings on desktop put in dock and set Open at login
          Cam1.evocamsettings
          • 320 * 180
          • 384 * 216
          • 480 * 270
          • font size 12
          • framerate 15
          • quality normal normal
          • fontsize 9
        • Cam2.evocamsettings
          • 1280 * 720
          • framerate 15
          • quality normal normal
        • Other resolutions 16*9
        • 256 * 144 -> YouTube 144p
        • 320 * 180
        • 480 * 270
        • 640 * 360 -> nHD
        • 768 * 432
        • 800 * 450
        • 848 * 480
        • 896 * 504
        • 960 * 540 -> qHD
        • 1024 * 576
        • 1152 * 648
        • 1280 * 720 -> HD
        • 1366 * 768 -> WXGA
        • 1600 * 900 -> HD+
        • 1920 * 1080 -> Full HD
        • 2000 * 1125
        • 2048 * 1152
        • 2304 * 1296
        • 2560 * 1440 -> QHD
        • 2880 * 1620
        • 3200 * 1800 -> QHD+
        • 3520 * 1980
        • 3840 * 2160 -> 4K UHD
        • 4096 * 2304 -> Full 4K UHD
        • 4480 * 2520
        • 5120 * 2880 -> 5K UHD
        • 5760 * 3240
        • 6400 * 3600
        • 7040 * 3960
        • 7680 * 4320 -> 8K UHD
        • 15360 * 8640 -> 16K

    Online references

    In this table are the commands for apache, bind and certbot via brew as in https://getgrav.org/blog/macos-mojave-apache-multiple-php-versions.

    How this page is made