Protected: SecureSFTP with 2Factor Auth

This content is password protected. To view it please enter your password below:

Posted in ipv6 | Enter your password to view comments.

Loading an OpenSSH Hostkey From a Hardware Token on FreeBSD

I had a requirement for creating an sftp server that needs strong client and host authentication.  The host needs to know its an authorized client connection, and the client needs to know its really the host its connecting to. SSH and public key crypto is great for this, but what if someone steals a copy of your private key ? What if someone breaks into your host and makes off with your hostkey ?  Until you detect the compromise and revoke and regenerate keys, you run the risk of a man in the middle attack among other things.
One way to mitigate this risk is by keeping your private keys on a hardware token on both sides.

Test setup: FreeBSD RELENG 10, Aladin eToken 64k (old Style with pkcs15 support). From the ports, OpenCT, OpenSC.  I built them from the ports as I wanted OpenSC to use OpenCT as the driver to interact with the Safenet eToken.

Lets start by erasing the token and setting up a pkcs15 filesystem.  Note, you might need to initialize the eToken on a Windows box to start from scratch.  Dont use these PINs in production. They are there just as an example!

0{sftp}# pkcs15-init -E
Using reader with a card: Aladdin eToken PRO 64k
0{sftp}# pkcs15-init -C -P --pin 12345678 --puk 999999 -a 01 --label "server1" --so-pin 12345678 --so-puk 999999 -T
Using reader with a card: Aladdin eToken PRO 64k

Now lets generate the actual private and public key on the token itself. There are two ways you can do this. You can either generate the key off the token and then import it, or you can ask the token to generate it on its own hardware. I think there are caveats to both approaches. If your token dies a hardware death, or lets say a malicious employee or hacker decides to lock the token by too many bad guesses, you are SOL and will need to generate a new key, and then have the entailing fallout from that. Also, how good is the crypto on the token ? Everyone loves to beat up OpenSSL, but it is well vetted, and the RND in the *BSD world is very well vetted and understood. Can the same be said for the software on the token ? I am not sure either way.

0{sftp}# pkcs15-init -G rsa/2048 -a 01 --pin 12345678 --so-pin 12345678 -u decrypt,sign                   
Using reader with a card: Aladdin eToken PRO 64k

We now have an RSA pair of keys on the token– private and public. Lets read the actual public key in an ssh friendly format.

0{sftp}# pkcs15-tool -k
Using reader with a card: Aladdin eToken PRO 64k
Private RSA Key [Private Key]
        Object Flags   : [0x3], private, modifiable
        Usage          : [0x2E], decrypt, sign, signRecover, unwrap
        Access Flags   : [0x1D], sensitive, alwaysSensitive, neverExtract, local
        ModLength      : 2048
        Key ref        : 16 (0x10)
        Native         : yes
        Path           : 3f005015
        Auth ID        : 01
        ID             : b146eef6387d12dd3431c758666e18785235bb7b
        MD:guid        : {cf3b339c-1ad9-b7f4-75a8-c530137c8751}
          :cmap flags  : 0x0
          :sign        : 0
          :key-exchange: 0

0{sftp}# pkcs15-tool --read-ssh-key b146eef6387d12dd3431c758666e18785235bb7b
Using reader with a card: Aladdin eToken PRO 64k
ssh-rsa AAAAB3NzaC1yc2EAAAAFAL4a91UAAAEBAId3Qzp2kfa8CEcP7x4ooCPw99szSfJIT6MnRNYLK2KUP/TTuMY6qi6Y2KKSaKyDHpJj6BDPLQ4i+z535+N+iZ/9Vw9sJv70brmBGkNLq2CsRBENCJeMVapcG5hbCrnVsn/GiEgdSZzF9mxC4o9v+d2ScbEwKsr1X5FDCcMyWUrwM3ioggQHK4eqB3Wv0WBFo8oNYHqymXiGs5WQ9bF4Mlvpvwbk2mzQUbEtX1xaCK2ehpgtpfTyQVTfVTKfh+eAPGZSmO6DnpITFHt3EE2JLw/Ar+7ERXmbHToG1A7/cIMhGMfdVaTvgnWbtnTA74cnqojddNVGrZoGS5I9VmR/5a0=

Lets now use that key for the server. To setup our sftp server, I will create a separate instance listening on port 26. We use the stock OpenSSH config for now. We copy over all the default configs as well as the pre-existing ssh keys. Make the following changes to the config you copied over

0{sftp}# cp -pR /etc/ssh /etc/ssh-26
0{sftp}# diff -u ../ssh/sshd_config sshd_config
--- ../ssh/sshd_config  2015-05-15 18:32:43.945683898 -0400
+++ sshd_config 2015-05-20 09:25:30.834911943 -0400
@@ -14,7 +14,7 @@
 # Note that some of FreeBSD's defaults differ from OpenBSD's, and
 # FreeBSD has a few additional options.
-#Port 22
+Port 26
 #AddressFamily any
 #ListenAddress ::
@@ -25,10 +25,11 @@
 # HostKey for protocol version 1
 #HostKey /etc/ssh/ssh_host_key
 # HostKeys for protocol version 2
-#HostKey /etc/ssh/ssh_host_rsa_key
+HostKey /etc/ssh-26/
 #HostKey /etc/ssh/ssh_host_dsa_key
 #HostKey /etc/ssh/ssh_host_ecdsa_key
 #HostKey /etc/ssh/ssh_host_ed25519_key
+HostKeyAgent /root/etoken-agent
 # Lifetime and size of ephemeral version 1 server key
 #KeyRegenerationInterval 1h
@@ -55,6 +56,7 @@
 # The default is to check both .ssh/authorized_keys and .ssh/authorized_keys2
 #AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys2
+AuthorizedKeysFile /etc/ssh-26/authorized_keys/%u
 #AuthorizedPrincipalsFile none
@@ -146,3 +148,10 @@
 #      AllowTcpForwarding no
 #      PermitTTY no
 #      ForceCommand cvs server
+Match Group sftponly
+   ChrootDirectory %h
+   ForceCommand internal-sftp
+   AllowTcpForwarding no
+   PermitTunnel no
+   X11Forwarding no

Lets now create the public_key for the server

0{sftp}# pkcs15-tool --read-ssh-key b146eef6387d12dd3431c758666e18785235bb7b > /etc/ssh-26/
Using reader with a card: Aladdin eToken PRO 64k
0{sftp}# chmod 600 /etc/ssh-26/ 

When the server sees that its just the public key and not the private key, the daemon will look to the defined agent socket to do all the necessary private key transformations. So pick your socket location in a place on your server that only root has access to.
Next, we fire up the agent with the socket that the server expects to communicate with. We then add to the agent via the pkcs#11 interface, the path that will let the private key do its magic on the token.

0{sftp}# ssh-agent -a /root/etoken-agent
setenv SSH_AUTH_SOCK /root/etoken-agent;
setenv SSH_AGENT_PID 25563;
echo Agent pid 25563;
0{sftp}# setenv SSH_AUTH_SOCK /root/etoken-agent;
0{sftp}# ssh-add -s /usr/local/lib/
Enter passphrase for PKCS#11: 
Card added: /usr/local/lib/

We are now ready to start up the server. Initially, try and do it via debug mode

0{sftp}# /usr/sbin/sshd -d -f /etc/ssh-26/sshd_config
debug1: HPN Buffer Size: 65536
debug1: sshd version OpenSSH_6.6.1p1_hpn13v11 FreeBSD-20140420, OpenSSL 1.0.1m-freebsd 19 Mar 2015
debug1: key_parse_private2: missing begin marker
debug1: key_parse_private_pem: PEM_read_PrivateKey failed
debug1: read PEM private key done: type 
debug1: will rely on agent for hostkey /etc/ssh-26/
debug1: private host key: #0 type 1 RSA
debug1: rexec_argv[0]='/usr/sbin/sshd'
debug1: rexec_argv[1]='-d'
debug1: rexec_argv[2]='-f'
debug1: rexec_argv[3]='/etc/ssh-26/sshd_config'
debug1: Bind to port 26 on ::.
debug1: Server TCP RWIN socket size: 65536
debug1: HPN Buffer Size: 65536
Server listening on :: port 26.
debug1: Bind to port 26 on
debug1: Server TCP RWIN socket size: 65536
debug1: HPN Buffer Size: 65536
Server listening on port 26.

In another session, lets just do a keyscan to see what the server serves up and see that it indeed matches the public key that we know

% ssh-keyscan -t rsa  -p 26 localhost  
# localhost SSH-2.0-OpenSSH_6.6.1_hpn13v11 FreeBSD-20140420
localhost ssh-rsa AAAAB3NzaC1yc2EAAAAFAL4a91UAAAEBAId3Qzp2kfa8CEcP7x4ooCPw99szSfJIT6MnRNYLK2KUP/TTuMY6qi6Y2KKSaKyDHpJj6BDPLQ4i+z535+N+iZ/9Vw9sJv70brmBGkNLq2CsRBENCJeMVapcG5hbCrnVsn/GiEgdSZzF9mxC4o9v+d2ScbEwKsr1X5FDCcMyWUrwM3ioggQHK4eqB3Wv0WBFo8oNYHqymXiGs5WQ9bF4Mlvpvwbk2mzQUbEtX1xaCK2ehpgtpfTyQVTfVTKfh+eAPGZSmO6DnpITFHt3EE2JLw/Ar+7ERXmbHToG1A7/cIMhGMfdVaTvgnWbtnTA74cnqojddNVGrZoGS5I9VmR/5a0=

Lets create the user now to ssh in. In production do the same pkcs15 key generation on the client’s hardware token. But for this example, we will use a traditional ssh key file

0{sftp}# pw groupadd sftponly
0{sftp}# pw useradd testuser1 -g sftponly -m
0{sftp}# chown root /home/testuser1
0{sftp}# mkdir /home/testuser1/files
0{sftp}# chown testuser1 /home/testuser1/files
0{sftp}# chflags schg /home/testuser1

We create the user and add them to the sftp only group. We ask the user for their public key, and we place it in the directory /etc/ssh-26/authorized_keys directory.

$ sftp -P 26
The authenticity of host '[]:26 ([]:26)' can't be established.
RSA key fingerprint is 58:59:a9:09:3c:c5:92:91:60:dc:d9:f5:0d:d7:92:95.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[]:26' (RSA) to the list of known hosts.
Enter passphrase for key '/home/testuser1/.ssh/id_rsa': 
Connected to
sftp> dir
sftp> pwd
Remote working directory: /

On the server, we check

0{sftp}# ssh-keygen -lf 
2048 58:59:a9:09:3c:c5:92:91:60:dc:d9:f5:0d:d7:92:95 (RSA)
Posted in freebsd | Tagged , , , , , , , | Leave a comment

Egypt is back

Nice to see my family online again!!

> sh ip bgp
BGP routing table entry for
Paths: (3 available, best #1, table Default-IP-Routing-Table)
  Advertised to non peer-group peers:
  174 3356 8452, (aggregated by 8452 from (
      Origin IGP, metric 12012, localpref 100, weight 112, valid, external, atomic-aggregate, best
      Community: 174:21000 174:22013
      Last update: Wed Feb  2 04:43:00 2011

  6453 3356 8452, (aggregated by 8452 (metric 1) from (
      Origin IGP, localpref 100, weight 110, valid, internal, atomic-aggregate
      Last update: Wed Feb  2 04:42:47 2011

  6453 3356 8452, (aggregated by 8452 (metric 1) from (
      Origin IGP, localpref 100, weight 111, valid, internal, atomic-aggregate
      Last update: Wed Feb  2 04:42:47 2011

Posted in bgp | Leave a comment

Novatel MC547 on FreeBSD 8.2

Not really an IPV6 posting, but I am curious how fast this will appear in google. (I will file a format PR later). But for users of the Novatel MC547 3g stick from Bell Canada, a bit of changes need to be made to the FreeBSD source code in order to get it to work.

diff  -u usbdevs.orig usbdevs
--- usbdevs.orig        2011-01-17 10:30:36.000000000 -0500
+++ usbdevs     2011-01-17 11:33:06.000000000 -0500
@@ -2388,6 +2388,7 @@
 product NOVATEL ZEROCD2                0x5030  Novatel ZeroCD
 product NOVATEL U727_2         0x5100  Merlin U727 CDMA
 product NOVATEL U760           0x6000  Novatel U760
+product NOVATEL MC547          0x7042  Novatel MC547
 product NOVATEL MC760          0x6002  Novatel MC760
 product NOVATEL2 FLEXPACKGPS   0x0100  NovAtel FlexPack GPS receiver

--- u3g.c       2010-12-02 21:34:53.000000000 -0500
+++ /tmp/u3g.c  2011-01-18 13:48:23.000000000 -0500
@@ -65,7 +65,7 @@

#ifdef USB_DEBUG
-static int u3g_debug = 0;
+static int u3g_debug = 1;

SYSCTL_NODE(_hw_usb, OID_AUTO, u3g, CTLFLAG_RW, 0, "USB 3g");
SYSCTL_INT(_hw_usb_u3g, OID_AUTO, debug, CTLFLAG_RW,
@@ -302,6 +302,7 @@
+       U3G_DEV(NOVATEL, MC547, 0),
U3G_DEV(NOVATEL, U720, 0),

camcontrol eject pass0 in some cases will put it in the correct modem mode after that. For the APN, I used and the following ppp.conf seems to work just fine on FreeBSD 8.x

set device /dev/cuaU5.0
set speed 921600
set timeout 0
set authname wapuser1
set authkey wap
set dial "ABORT BUSY TIMEOUT 2 \
\"\" \
AT+CGDCONT=1,\\\"IP\\\",\\\"\\\" OK \
AT&v OK \
set crtscts on
#set mtu maximum 296
#set mru maximum 296
disable vjcomp
disable acfcomp
disable deflate
disable deflate24
disable pred1
disable protocomp
disable mppe
disable ipv6cp
disable lqr
disable echo
nat enable yes
disable dns
resolv writable
set dns
set ifaddr
#add default HISADDR          # See*

…. PR created

Posted in freebsd | Tagged , , , , | 1 Comment

Missing routes

In IPv4 land, you could probably get by with a single feed and see the entire internet.  Of course you would not have any redundancy, but apart from the odd spat and peering fight, you could get to everything.  In V6, not so much.  Of the 3 transit peers I have, the numbers range quite a bit.  AS174 is missing about 20% of what AS6939 and AS6453 has. There is idle gossip on as to the reasons… but there are still some 500 /32s (ignoring more specific /48s which might be reachable through the /32)


  11647 6453 293
  11647 6453 701 668
  11647 6453 30071 13645
  11647 13030 15716
  11647 6453 5511
  11647 6453 6762 5609, (aggregated by 5609
  11647 6453 6830
  11647 6453 25137
  11647 6453 30071 4608, (aggregated by 4608
  11647 6453 30071 2549
  11647 6453 30071 10318
  11647 6453 6762 7303
  11647 6453 30071
  11647 6453 6762 8280
  11647 6453 13030
  11647 13030
  11647 6453 701
  11647 6453 6762
  11647 6453 5511 8346
  11647 6453 30071
  11647 6453 13030 8271
  11647 13030 8271
  11647 6453 13030 33845
  11647 6453 30071 4608, (aggregated by 4608
  11647 6453 701 18061 9555
  11647 6453 6762 7642
  11647 6453 30071 6536
  11647 6453 701 18750
  11647 6453 30071 19151
  11647 6453 701 26773
  11647 6453 30071 10326
  11647 6453 30071 19151 16842
  11647 6453 30071 19151 31877
  11647 6453 30071 19151 22911
  11647 6453 30071 13911
  11647 6453 30071 7786
  11647 6453 30071 13911 14595
  11647 6453 6762 7303 4270
  11647 6453 6762 7303 4270 27770
  11647 6453 6762 7303 4270 5692
  11647 6453 13030 48218
  11647 13030 48218
  11647 6453 13030 20634
  11647 13030 20634
  11647 6453 701 12702 24807
  11647 6453 6830
  11647 6453 5511 8697
  11647 6453 6762 31463
  11647 13030 9191
  11647 6453 13030 25164
  11647 13030 25164
  11647 6453 13030 16242
  11647 13030 16242
  11647 6453 13030 28717
  11647 6453 13030 25563
  11647 13030 25563
  11647 6453 5511 3215
  11647 6453 5511 3215
  11647 6453 5511 3215
  11647 6453 5511 12493
  11647 6453 13030 44573
  11647 6453 13030 35366
  11647 6453 13030 29430
  11647 13030 29430
  11647 6453 13030 21232
  11647 13030 21232
  11647 6453 13030 47617
  11647 13030 47617
  11647 6453 6830 20825
  11647 6453 6762 8953
  11647 6453 13030 15216
  11647 13030 15216
  11647 6453 13030
  11647 13030

For a full list of /32 routes, see ipv6-routes-missing.txt

BTW, there is a cool tool at which can give further information/views

Posted in bgp, ipv6 | Leave a comment


Well, it sounds dire, but
it will be a while before networks are forced to move to ipv6. Still, it will make for good headlines 🙂

I think the next phase will be various registries turning up the financial heat on existing block owners to demonstrate real and proper usage of their existing allocations first….

NetRange: –
NetName:        APPLE-WWNET
NetHandle:      NET-17-0-0-0-1

NetRange: –         APPLE-WWNETNetHandle:      NET-17-0-0-0-1Parent:

Apple ? You really need a /8 ?  Same with HP and a few others.

Posted in ipv6 | Tagged | Leave a comment

ipv6 PTR Records and Mail

Ran into a small issue with a colleague’s email getting flagged as spam.  SpamAssassin, didnt like the fact that his PC (which defaults to IPv6 if available) didnt have  PTR record.  As Windows 7 has IPv6 privacy extensions on by default there was no PTR record to match to an A record.  Generally this was a no-no in ipv4 days, but in IPv6 days, its err… not practical. To quote

[to create a /48 worth of ptr records] If 1000 entries could
 be written per second, the zone  would still not be complete
 after two quintillion years.

OK, so thats not gonna happen here….And asking all outside mail servers to “fix” this check is not going to happen there either 🙁  Dont really like many of the options listed in the RFC either… Dynamic DNS, hacking up DHCP6 (What happened to the benefits of SAC?)…. Hmmmm

Posted in ipv6 | Leave a comment

Not quite naked without my NAT but feeling the draft

One of the promises of IPv6 is that its supposed to do away with the evils of NAT! My Favorite book so far on IPv6 is “IPv6 in Practice” and he writes on one of the benefits of IPv6

Abolition of NAT With IPv6 there is no need to connect multiple machines to the Internet using a single address and network address translation (NAT). Without NAT, end-to-end connectivity becomes available again, allowing machines to connect to each other without intermediate “broker” services, like mail exchangers/relays, web proxies, DNS forwarders or SIP gatekeepers, that are run by a service provider. At first glance this doesn’t seem like much of an advantage, but at this time its consequences are barely fathomable, making services possible that are difficult even to imagine to our NAT-conditioned minds.

Allowing… end to … end connectivity ??? Good lord. I dont want unrestricted end to end connectivity. I want it restricted on the big bad internet.  And the strawman argument that NAT is not security that you will often see in other IPv6 discussions is just that, a strawman. .  Of course, NAT by itself is NOT security.  But given the choice between

a) Stateful firewall between hosts and the internet
b) Stateful firewall between hosts that are RFC 1918 and the internet

I will take b)
Case in point. While setting up our office firewall to allow out IPv6 from our workstations, I had allowed all icmp6 traffic in and out by accident. BAD! Is it the end of the world, no of course not. It would take eons for someone to scan for internal hosts with a traditional ping scan. But outbound connectivity from various internal severs will leave enough information for someone to find such hosts by other means. e.g. our internal WINS file server has a static IPv6 IP. It also punts out mail on occasion… which will be in some headers. With that in mind, someone could now send icmp6 traffic to the host. Is that a bad thing… Well, not in of itself, but hacks rarely are. They are usually a combination of partial vulnerabilities combined together in novel ways.
Looking at my favorite OS, FreeBSD, I see the following default rules for IPv6 ipfw

00400           0              0 deny ip from any to ::1
00500           0              0 deny ip from ::1 to any
00600           8            580 allow ipv6-icmp from :: to ff02::/16
00700      118703        8071672 allow ipv6-icmp from fe80::/10 to fe80::/10
00800        4890         507228 allow ipv6-icmp from fe80::/10 to ff02::/16
00900       21025        3182910 allow ipv6-icmp from any to any ip6 icmp6types 1
01000      179147       12264112 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136

OK, so some link local prefixes, fair enough… icmp types 1,2,135 and 136. Looking at the sys/netinet/icmp6.h, it looks like they are

#define ICMP6_DST_UNREACH               1       /* dest unreachable, codes: */
#define ICMP6_PACKET_TOO_BIG            2       /* packet too big */
#define ND_NEIGHBOR_SOLICIT             135     /* neighbor solicitation */
#define ND_NEIGHBOR_ADVERT              136     /* neighbor advertisement */

Ummm, ok. I guess that makes sense. But why not

#define ICMP6_TIME_EXCEEDED             3       /* time exceeded, code: */
#define ICMP6_PARAM_PROB                4       /* ip6 header bad */

They seem reasonable messages you would want to allow too no ? Not a rhetorical question….

Posted in freebsd, ipv6 | Tagged , , , , | 1 Comment


More notes after the fact…. It was surprisingly easy actually. Started out on Torix.  Most everything locally can be had via the route servers.  One small annoying thing… Who the hell came up with

show ipv6 bgp sum

Whoever decided that syntax has either super long fingers, or does not type that combo of keys (v then 6) very much…  Try it…. ipv6… ipv6.  When you are debugging pathing issues, you have to type it a lot and its very annoying!

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
                4 yyyyy   22512   70943        0    0    0 02w1d15h        2
                4 yyyyy   26464   22514        0    0    0 02w1d15h      250
                4  yyyy  107913   23254        0    0    0 02w1d15h      141
                4  7081   67602  116045        0    0    0 02w1d15h       35
                4  7081   67605  116276        0    0    0 02w1d15h       35

Total number of neighbors 6

A full view is only about 3000 routes

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
                4  6453   55779    7908        0    0    0 5d11h43m     3168

In terms of config, there is not a hell of a lot more than plain old bgp4

 address-family ipv6
 network 2607:f3e0::/32
 neighbor 2001:5a0:1000:200::5 activate
 neighbor 2001:5a0:1000:200::5 remove-private-AS
 neighbor 2001:5a0:1000:200::5 soft-reconfiguration inbound
 neighbor 2001:5a0:1000:200::5 prefix-list SENTEX-INET6 out
 neighbor 2001:5a0:1000:200::5 route-map TELEGLOBE-6-OUT out

In terms of dealing with peers it was fairly straight forward. Cogent was very easy. “Hey, I want IPV6 transit”… “Sign this paper” and that was that. A week later I had the info and was ready to go. TATA (6453) was a lot more paperwork and back and forth, but once I dealt with the ipnoc folks it was easy enough as they were pretty clueful. The peers at Torix were also easy to deal with. is probably the most IPv6 gung ho org out there. They have a lot of tutorials and links aggregated into one location which I will write about later

Posted in bgp, freebsd, ipv6 | Tagged , , | Leave a comment


OK, added smtp mx records as well. I dont really use the domain for email, but I am sure my dog will chew up some spam. I wonder how long before he gets some spam from the crawlers….

0# host -tmx mail is handled by 50 mail is handled by 10
0# host has address has address has address has address has IPv6 address 2607:f3e0::9 has IPv6 address 2607:f3e0::4
Posted in freebsd, ipv6 | Tagged , | Leave a comment