Fix audio issues with Dell Latitude 5590

 - 30-04-2020

Last year I was playing around with a Dell Latitude 5580 and 5590 which had strange audio problems. With all drivers installed and updated (Dell Command | Update) it still was not playing clear audio. Searching Reddit, contacting the reseller and reaching out to Dell Support was a nice trip down the internet but I’ve managed to fix it with Dell’s ProSupport team.

Symptoms

The symptoms are clear as day: playing any kind of video or music will play distorted, crackle and pop. This is not per-se in some kind of logic pattern and is completely random but I have noticed that when you have many things running on the laptop it gets much worse. It also does not matter whether you are using headhpones (into the jack) or using the internal speakers.

Things tried

As this was very frustrating I started with Reddit and Google first. Tried all different kinds of audio drivers, BIOS updates, Windows reinstalls and Waves Audio application versions but nothing worked. Strangely when uninstalling the Realtek driver fixed it for that session but upon rebooting it was back as the driver is then reinstalled by Windows automatically. I’ve also contacted my local reseller and they have replaced the speakers and even the mainbord just to be sure but that did not help as well.

The solution

Then I decided to contact Dell ProSupport as this was available for these Latitudes and one of the first things they ask is whether you have updated the BIOS etc. But more strange was the question whether I had AHCI enabled for my harddisk or that Intel RST was enabled (Raid on mode). As this laptop only has space for 1 drive I’ve set it to AHCI as RAID would never be used. They asked me to change it to “Raid on” mode and try again and I was seriously baffled! It worked! My audio was playing loud and clear without any stuttering, popping and cracking! Upon asking why a harddisk setting would affect sound behaviour the answer was “The technicians use it as a workaround”. So that will remain a mystery but at least it’s solved.

I’ve seen lots of topics on the internet about this problem. While some may actually be caused by faulty drivers or hardware this may be a solution as well. It has been a while since I posted something but I hope it may serve alot of people in their search of fixing their audio problems :).

In Software, Tweaks

Enable Windows Hello on older Dell Latitudes

 - 13-06-2018

My previous post was about fixing firmware issues on a Latitude E6320. On said machine is also a fingerprint sensor available. As this allow faster and a more secure login this is a method that I definitely want to use.

Windows Hello
The problem is that with all drivers installed available from Dell, fingerprint sign in is still not available for use with Windows Hello. Windows Hello is a login framework that allows other types of logins besides passwords and pins. So you can use fingerprints in this example or smartcards. Windows 10 and Windows Hello has a minimum requirements list but it’s not clear what this list is and mostly means that any laptop / machine older than a couple of years will not fit within these requirements. This is quite ridiculous to me as it’s just software and should be able to benefit from it either way!

Here you see that Windows Hello is not available on my E6320 yet I have all ControlVault related items installed on my machine.

Installing USH / ControlVault software:
In order to use the fingerprint reader for sign-in you need to install several pieces of software. Make sure that you at least install the ControlVault driver which you can download from Dell directly here.

You would think that this will install all software related to the USH chip and hardware components but this is not true! You will also need to install the ControlVault Biometric Framework driver. You can download this as well from Dell directly here.

After installing the 2 components give the machine a reboot. Afterwards open the program called “Control Vault WVF”. A window like below will open:

Select “Enable” and click on “Set”. Now open the Windows Hello settings page again and see that Windows Hello now is available (see below)!

Now you can set up fingerprint based access!

I have noticed that the fingerprint reader on my E6320 is rather slow and has a delay of a second before responding. I don’t know whether this is hardware or software related but I don’t mind this little delay.

In Software, Tweaks, x86

Fixing corrupt Dell USH / TPM firmware

 - 08-06-2018

I have an old Dell Latitude E6320 and dates back from 2011. While the machine itself is ancient for IT standards it still functions pretty well and based on it’s 13″ size it is very suitable for traveling and meetings.

The machine came in my possession last year and had never seen any BIOS or other firmware updates so that was the first thing I updated on this laptop. There is also a security device in the laptop (Broadcom USH) which contains the TPM, Fingerprint sensor and smartcard reader devices.

Updating the USH firmware has failed:
When updating the USH firmware the laptop froze at the part where the TPM was flashed. I left the machine alone for around a couple of hours but nothing responded anymore. I deciced to hard reset the machine and that is where the issues started on the machine.

Symptoms:
With the firmware update that was only half-processed and a TPM chip that was not working anymore (due to the failed update the BIOS did not see the TPM anymore…) other problems started to occur as well. When powering on the laptop or when booting Windows it would randomly shutdown. The fan starts blowing at full speed and the status leds left of the keyboard started showing a diagnostic pattern (as is usual with Dell’s). The pattern was:

HDD: Blinking
Battery: Solid
WIFI: Solid

Based on the technical documentation from Dell this is described as “A possible processor failure”. What causes it are some obvious symptoms as overheating, CPU not seated well in it’s socket and other symptoms. But as the CPU is soldered on the mainboard in the E6320 this could not be it and had to be related to the failed firmware update.

Also in the BIOS, the TPM cannot be activated. The checkbox for “Clear” is permanently checked and therefore prohibits you from using the TPM and will not advertise it to the system / OS.

Fixing the issue:
It took a while before I saw that the firmware update that you run on Windows can also be used for DOS. So I created a FreeDOS bootable USB-stick and copied the whole update folder to the USB-stick and rebooted into FreeDOS.

From there I ran the “DOSUPDATE.BAT” script and the script starts spewing out alot of information about the progress. The BMC update may take 2 to 3 minutes to complete, after that the TPM was flashed (with succes in this case!) and returns to the prompt.

When I rebooted the machine and went back into the BIOS I could activate the TPM again and the “Clear” box was not checked anymore and Windows saw the TPM device again! Also the strange behaviour with the random shutdown was fixed!

At the end of the update I got an error about the PBA that could not be updated. You will see a notice stating that you neet to clear the CV Admin and try again. The PBA (Pre Boot Authentication, a feature that sets a password on the boot process) is not a feature that I use and is not enabled on my system so I did not bother to check that out.

In Software, x86

Fixing the 3-beep issue on a Dell Inspiron 15z

 - 30-05-2017

Recently a Dell Inspiron 15z 5523 was handed over to our IT department for disposal. The issue with the laptop was that upon powering on it will beep 3 times and continuously to do so. While the laptop itself still looked pretty good (it’s an ultrabook) I wanted to have a look at the machine for myself so I took the unit home.

Specifications:
As I always do I will reveal the specs below. Note that the machine is from 2013 and therefore a couple of years old. The harddisk and memory was already stripped from the machine, so I placed some spare parts of my own in it to have a working whole.

CPU: Intel Core i5 3337U (2 x 1.8GHz with HT and turbo to 2.7GHz)
GPU: Integrated HD4000 graphics
RAM: 2 x 4GB DDR3 @ 1600MHz
HDD: 128GB mSATA SSD
ODD: Tray loading super-multi drive
USB: 2 x USB3.0 and 2 x USB2.0
NET: Atheros 1Gbit LAN and Intel WiFi 2230
OTHER: 1 x HDMI, 1 x SD reader, media buttons and a headphone jack

The screen is 15″ with a resolution of 1366×768 and has a capacitive touchscreen.

3 beep issue:
Dell uses beeps to indicate that there may be something wrong with the system. Where 2 beeps equals to a bad RAM module, the 3 beeps indicate a general system failure. As a system failure can be interpreted on a wide scale it’s not a narrow pin-point system what is actually failing on the system board. For instance it could be a corrupt bios, low CMOS battery etc.

Searching the internet on this issue I found out that it seems to be a very common habit for the Inspiron 15z to have the 3-beep issue. A lot of people are having issues as soon as they upgrade to Windows 10. While the 3-beep issue may be true to a system board failure (I’ve seen enough of them in the past) I had the feeling that it may be BIOS or software related. Upon further searching it seems that Dell has released a BIOS update for this machine with revision A05 which addresses this issue (for the Inspiron 13z it’s A10 or higher).

Prerequisites:
You will need some tools and software in order to continue:

* Philips screwdrivers to remove the screws (small ones)
* The BIOS update file (link here)
* FreeDOS (link here)
* A USB keyboard
* 30 minutes of your time

You need to create a USB bootable media from the FreeDOS image and place the BIOS update executable on it.

The steps to create the stick can be found all over the internet and will therefore not be covered in this post.

Opening up the laptop:
You need to disassemble almost the whole machine in order to reach the CMOS battery or other internal parts. As is with most Dell consumer laptops you have to remove the bottom screws (7x), the memory cover (because that unlocks the optical drive), remove the optical drive to reveal 1 very flat screw and within the memory compartiment there is one screw marked with “KB” which holds down the keyboard.

Turn over the machine and remove the keyboard, again 4 screws are waiting to be removed and several ZIF-cables have to be released.

Once that’s finished you can gently pry open the case, I found this the easiest to start at the optical drive opening as it released most easily there.

Once you have the cover removed the inside looks like this:

You can see the CMOS battery right beneath the Inspiron logo along with the WiFi module. There is also a little switch right of the HDMI connector which cut’s down the power from the battery if switched to the right and we need to switch that later.

Resetting the CMOS:
The most used trick to solve this issue is to reset the CMOS. In order to do so you have to:
* Remove the power cord from the machine
* Switch the battery power switch to the right
* Remove the CMOS battery

Now wait for a couple of minutes, so we are sure the machine is completely “dead”. After this, perform the steps above but this time in reverse order (and have move the switch to the left position).

Powering up the machine:
Now plug in the USB keyboard and press the power button on the machine (it’s beneath the left screen hinge). In my case the machine showed the DELL logo after 10 to 15 seconds. Press F2 repeatedly to enter the system BIOS.

Here I saw that the machine had a older BIOS than the current one by DELL which addresses the issue, in my case it was A02.

Updating the BIOS:
Now that the machine powers up again we need to update the BIOS as fast as possible to prevent it from going corrupt again (and you have to start over, I had to do so several times due to tests).

Insert the USB bootable media which contains FreeDOS and the BIOS update file and reset the machine by pressing Control + Alt + Delete. When you see the DELL logo start pressing F12 to have the boot menu pop up. Select the USB stick as boot device and press enter. You have to boot into FreeDOS without any memory drivers / expanders for the update to work. Start the executable to perform the update:

It may take several minutes to complete and the fan start running at full speed. Do not remove power from the machine during the update as this will permanently brick the device! After this the machine will reboot and start booting again.

After the update you can assemble the machine and should be good to go!

The battery needs at least 10% charge in order to update the BIOS whether it’s connected to a wall outlet or not.

Conclusion:
Well, pictures don’t lie, here is a picture of the machine after the upgrade with Windows 10 on it:

One thing I did notice immediately was that after the upgrade the machine powered up much faster. Previously I had to wait 10 to 15 seconds before even the DELL logo showed. After the upgrade this was almost instant!

HELP! It did not work!
To be honest here: I can’t help you further then. While this trick works for most of the people with this issue it may still be a faulty mainboard which needs replacement. Should you have any questions regarding this issue or have fixed the issue with my information, feel free to leave a comment!

In x86

Updating the firmware on a 3Ware 9650SE controller via CLI

 - 29-11-2016

8336283143wareEarlier this year I acquired an old SuperMicro storage server which sports a 3Ware 9650SE-16ML RAID-controller. I sometimes see that it randomly starts throwing “DEVICE-ERROR” errors in the tw-cli tool. Mostly these appear when the controller is too busy or when a harddrive is responding too slow and are false positives. As I monitor the disks in this system I receive notifications about this and is not desirable when they are false.

First thing I checked was the firmware of the controller which seems to date back to 2009/2010. The controller is considered legacy as of 2015 and will not receive further firmware updates from Avago. As a result the latest available firmware is from 2014/2015 which we will install in this post.

Prerequisites:
The instructions will be performed via SSH as the root user. You need the “tw-cli” tool installed to communicate with the 3Ware RAID-controller. The tool can be downloaded from the Avago website or hwraid.le-vert.net. The firmware is also required and can be downloaded from the Avago site as well. For the 9650SE card the firmware will be downloaded from the CLI directly (as I know the URL).

Getting current firmware version:
When logged in as root via SSH run the following to retrieve the card’s slot:
tw-cli show

The output in our case is:
Ctl Model (V)Ports Drives Units NotOpt RRate VRate BBU
------------------------------------------------------------------------
c2 9650SE-16ML 16 15 1 0 1 1 -

The slot the RAID-controler resides in is “c2” in this case. Now we will get the firmware version of the card:
tw-cli /c2 show firmware

Which will show similar output (version may be different) as below:
/c2 Firmware Version = FE9X 4.08.00.008

Preparing the firmware update:
We need to download the firmware package for the RAID-controller. I know the URL for the firmware that is needed for my card, so I can simply use wget for it. For your card, download and upload it to your server.

To download the firmware package for the 9650SE line of cards use:
wget http://docs.avagotech.com/docs-and-downloads/raid-controllers/raid-controllers-common-files/9650SE_9690SA_firmware_9-5-5-1codeset_fw4-10-00-027.zip

Unpack the firmware files:
unzip 9650SE_9690SA_firmware_9-5-5-1codeset_fw4-10-00-027.zip

This will leave several files in the directory where the ZIP file is in. The one we need to flash is called “prom0006.img”. This information can be retrieved by looking into the up9650.bat file which points to the prom0006.img file.

In my case the absolute path to the file is “/root/prom0006.img”. You need to know the absolute location to the firmware file for use in tw-cli later on.

Updating the firmware:
Now that we have the firmware image, we can start flashing it into the RAID-controller with the following command:
tw-cli /c2 update fw=/root/prom0006.img

This will show the following output, waiting for your confirmation:
Warning: Updating the firmware can render the device driver and/or
management tools incompatible. Before you update the firmware,
it is recommended that you:

1) Back up your data.

2) Make sure you have a copy of the current firmware image so that
you can roll back, if necessary.

3) Close all applications.

Examining compatibility data from firmware image and /c0 ... Done.

New-Firmware Current-Firmware Current-Driver Current-API
------------------------------------------------------------------------
FE9X 4.10.00.027 FE9X 4.08.00.006 2.26.02.014 2.08.00.023

Both API and Driver are compatible with the new firmware.
Recommendation: proceed to update.

Given the above recommendation...
Do you want to continue ? Y|N [N]: Y
Downloading the firmware from file /root/prom0006.img ... Done.
The new image will take effect after reboot.

Check if the recommendation is set to “proceed to update” and confirm with Y.
Downloading the firmware from file /root/prom0006.img ... Done.
The new image will take effect after reboot.

Now that the update is finished, reboot the machine to have the new firmware activated in the RAID-controler:
reboot

Checking new installed firmware version:
Once the server is back online, check the firmware version again to confirm the update went with success:
tw-cli /c2 show firmware
/c2 Firmware Version = FE9X 4.10.00.027

As seen above, the update of the firmware went with succes!

In Server, Software

Reinstalling a Juniper SRX100 with corrupt flash via CLI

 - 12-09-2016

srx100At work we have several old SRX100H’s lying around which are not able to boot anymore. I took one home last week to see if I could resurrect the unit and make it operational again. Based on the specs of the device it would make a very nice router/firewall. The problem is that the device will not boot and simply will not start JunOS (the OS on the device). Hooking up the console shows that the device gets a kernel panic as the internal flash disk has filesystem issues. This is a common issue on the SRX100 and mainly occurs when the power is plugged from the device while it’s in operational state. This post will explain what steps needs to be taken to revive the unit.

Specifications:
CPU: Octeon CN5020 dualcore MIPS64 @ 500MHz
RAM: 1GB
LAN: 8x 100Mbit
MGT: 1x serial (RJ45) @ 9600 baud
OTHER: Has USB port for installation and recovery. Also has full UTM functions with appropriate licensing (not default)

The CPU has offloading support for various common used ciphers to speed up encryption tasks.

Requirements:
For the recovery I will use TFTP as I already have this configured on my network. You also need access to the firmware files within the Juniper download center which means you need to be a valid Juniper partner in order to access the firmware. I found a collegue of mine which has access to this area and downloaded the firmware for me. You will also need a serial cable for managing the device, something like a RJ-45 -> DB9 (null modem cable).

Recovery via USB:
Recovery via USB is also possible. For this a USB-stick of at least 256MB is needed and formatted with the FAT32 filesystem. Just copy the firmware file to the USB-stick. You can skip the networking setup and go straight to the Starting installation part. Make sure that you set up the console connection though!

Getting the firmware:
This is the hardest part as you may only download firmware updates for Juniper devices if you have a support plan with Juniper. As I personally don’t have a contract with them I’ve found a colleague of mine who normally manages this for the company and downloaded the latest firmware updates for me. If you don’t have a contract with Juniper it’s not possible to download the firmware. You should contact a network engineer in your company or personal area to see if they can help you further. At the time of writing version “JunOS 12.1X46-D55” was the latest release with a size of approx ~150MB.

A newer release is available as well which starts with JunOS 12.3X… but that is not available for the SRX100H and SRX100B models. So here JunOS 12.1X46-D55 is the latest firmware update that is available for these models.

Connecting the switch via serial:
Now connect the switch to your serial console. 9600 baud 8N1 should do the trick to communicate with the switch. Turn the switch on after you’ve connected the console as you have to halt the 3 second autoboot process!

You should see something like below:
U-Boot 1.1.6-JNPR-2.0 (Build time: Nov 17 2010 - 07:04:52)

SRX_100_HIGHMEM board revision major:0, minor:0, serial #: AT0812AF0793
OCTEON CN5020-SCP pass 1.1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)
DRAM: 1024 MB
Starting Memory POST...
Checking datalines... OK
Checking address lines... OK
Checking 512K memory for U-Boot... OK.
Running U-Boot CRC Test... OK.
Flash: 4 MB
USB: scanning bus for devices... 3 USB Device(s) found
scanning bus for storage devices... 1 Storage Device(s) found
Clearing DRAM........ done
BIST check passed.
Boot Media: nand-flash usb
Net: pic init done (err = 0)octeth0
POST Passed
Press SPACE to abort autoboot in 1 seconds
=>

Preparing the downloaded firmware:
If you have downloaded the firmware you should end up with a .tgz file. You don’t have to extract it and can copy it directly to servers your TFTP folder! The installer will extract it for you during installation.

Setting network parameters for installation:
Before we can start loading the firmware from the network we need to set the IP parameters in uBoot so the installer can reach your TFTP server. My TFTP server is at 192.168.0.75 and I’ve given the Juniper for this install the IP 192.168.0.210. My gateway is traditionally at 192.168.0.1:
setenv ipaddr 192.168.0.210
setenv serverip 192.168.0.75
setenv gatewayip 192.168.0.1

Save your settings:
saveenv
Saving Environment to Flash...
Un-Protected 1 sectors
Erasing Flash...
. done
Erased 1 sectors
Writing to Flash... writing to flash...
done
Protected 1 sectors

Starting the installation:
Now reset the device via uBoot:
reset

And wait until you see the following output with especially the last line:
reset

U-Boot 1.1.6-JNPR-2.0 (Build time: Nov 17 2010 - 07:04:52)

SRX_100_HIGHMEM board revision major:0, minor:0, serial #: AT0812AF0793
OCTEON CN5020-SCP pass 1.1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)
DRAM: 1024 MB
Starting Memory POST...
Checking datalines... OK
Checking address lines... OK
Checking 512K memory for U-Boot... OK.
Running U-Boot CRC Test... OK.
Flash: 4 MB
USB: scanning bus for devices... 3 USB Device(s) found
scanning bus for storage devices... 1 Storage Device(s) found
Clearing DRAM........ done
BIST check passed.
Boot Media: nand-flash usb
Net: pic init done (err = 0)octeth0
POST Passed
Press SPACE to abort autoboot in 1 seconds
ELF file is 32 bit
Loading .text @ 0x8f000078 (244960 bytes)
Loading .rodata @ 0x8f03bd58 (13940 bytes)
Loading .rodata.str1.4 @ 0x8f03f3cc (16648 bytes)
Loading set_Xcommand_set @ 0x8f0434d4 (100 bytes)
Loading .rodata.cst4 @ 0x8f043538 (20 bytes)
Loading .data @ 0x8f044000 (5608 bytes)
Loading .data.rel.ro @ 0x8f0455e8 (120 bytes)
Loading .data.rel @ 0x8f045660 (136 bytes)
Clearing .bss @ 0x8f0456e8 (11656 bytes)
## Starting application at 0x8f000078 ...
Consoles: U-Boot console
Found compatible API, ver. 2.0

FreeBSD/MIPS U-Boot bootstrap loader, Revision 2.0
(builder@warth.juniper.net, Wed Nov 17 07:07:32 UTC 2010)
Memory: 1024MB
[9]Booting from nand-flash slice 3
Un-Protected 1 sectors
writing to flash...
Protected 1 sectors
\
can't load '/kernel'
can't load '/kernel.old'
Press Enter to stop auto bootsequencing and to enter loader prompt.

Here press the Enter key so you will be dropped at the loader prompt. You should be now left at the following prompt:
Type '?' for a list of commands, 'help' for more detailed help.
loader>

To start the installer via TFTP run the following command:
install tftp://192.168.0.75/junos-srxsme-12.1X46-D55.3-domestic.tgz

Should you want to start the installation from the USB-stick:
install file:///junos-srxsme-12.1X46-D55.3-domestic.tgz

Starting the installer may take several minutes as the archive is being copied to the device and extracted for further installation. Note that the output of the installer will be extremely long and has 2 stages where the 1st stage is the actual installation and the 2nd stage is the first startup of the device in which it will generate it’s host keys etcetera. Overall the installation time may vary from 15 ~ 30 minutes.

The first part looks like (prepare for flood!):
octeth0: Up 1000 Mbps Full duplex (port 0)
/kernel data=0xb16d5c+0x134b2c syms=[0x4+0x8bbd0+0x4+0xcadc3]
Kernel entry at 0x801000e0 ...
init regular console
Primary ICache: Sets 64 Size 128 Asso 4
Primary DCache: Sets 1 Size 128 Asso 64
Secondary DCache: Sets 128 Size 128 Asso 8
GDB: debug ports: uart
GDB: current port: uart
KDB: debugger backends: ddb gdb
KDB: current backend: ddb
kld_map_v: 0x8ff80000, kld_map_p: 0x0
Copyright (c) 1996-2016, Juniper Networks, Inc.
All rights reserved.
Copyright (c) 1992-2006 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
JUNOS 12.1X46-D55.3 #0: 2016-07-08 18:46:54 UTC
builder@quoarth.juniper.net:/volume/build/junos/12.1/service/12.1X46-D55.3/obj-octeon/junos/bsd/kernels/JSRXNLE/kernel
JUNOS 12.1X46-D55.3 #0: 2016-07-08 18:46:54 UTC
builder@quoarth.juniper.net:/volume/build/junos/12.1/service/12.1X46-D55.3/obj-octeon/junos/bsd/kernels/JSRXNLE/kernel
real memory = 1073741824 (1024MB)
avail memory = 559992832 (534MB)
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
Security policy loaded: JUNOS MAC/pcap (mac_pcap)
Security policy loaded: JUNOS MAC/runasnonroot (mac_runasnonroot)
netisr_init: !debug_mpsafenet, forcing maxthreads from 2 to 1
cpu0 on motherboard
: CAVIUM's OCTEON 5020 CPU Rev. 0.1 with no FPU implemented
L1 Cache: I size 32kb(128 line), D size 8kb(128 line), sixty four way.
L2 Cache: Size 128kb, 8 way
obio0 on motherboard
uart0: on obio0
uart0: console (9600,n,8,1)
twsi0 on obio0
dwc0: on obio0
usb0: on dwc0
usb0: USB revision 2.0
uhub0: vendor 0x0000 DWC OTG root hub, class 9/0, rev 2.00/1.00, addr 1
uhub0: 1 port with 1 removable, self powered
uhub1: vendor 0x0409 product 0x005a, class 9/0, rev 2.00/1.00, addr 2
uhub1: single transaction translator
uhub1: 2 ports with 1 removable, self powered
umass0: STMicroelectronics ST72682 High Speed Mode, rev 2.00/2.10, addr 3
cpld0 on obio0
pcib0: on obio0
Disabling Octeon big bar support
PCI Status: PCI 32-bit: 0xc041b
pcib0: Initialized controller
pci0: on pcib0
pci0: at device 2.0 (no driver attached)
pci0: at device 2.1 (no driver attached)
pci0: at device 2.2 (no driver attached)
gblmem0 on obio0
octpkt0: on obio0
cfi0: on obio0
octpkt_attach: Initializing octpkt0 interface
Timecounter "mips" frequency 500000000 Hz quality 0
###PCB Group initialized for udppcbgroup
###PCB Group initialized for tcppcbgroup
md0: Preloaded image 10022912 bytes at 0x80ea2224
da0 at umass-sim0 bus 0 target 0 lun 0
da0: Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 1000MB (2048000 512 byte sectors: 64H 32S/T 1000C)
Trying to mount root from cd9660:/dev/md0
WARNING: preposterous time in file system
WARNING: clock 10662 days greater than file system time
tty: not found
Starting JUNOS installation:
Source Package: net0:/junos-srxsme-12.1X46-D55.3-domestic.tgz
Target Media : internal
Product : srx100h
add default: gateway 192.168.0.1
PING 192.168.0.75 (192.168.0.75): 56 data bytes
64 bytes from 192.168.0.75: icmp_seq=0 ttl=64 time=1.493 ms

--- 192.168.0.75 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.493/1.493/1.493/0.000 ms
Computing slice and partition sizes for /dev/da0 ...
Media check on da0
Attempting to save existing configuration...
** /dev/da0s3e
Cannot find file system superblock

LOOK FOR ALTERNATE SUPERBLOCKS? yes

32 is not a file system superblock
USING ALTERNATE SUPERBLOCK AT 12768
** Last Mounted on
** Phase 1 - Check Blocks and Sizes
UNKNOWN FILE TYPE I=2
CLEAR? yes

Here you can see that the internal filesystem is corrupt. In my case this CLEAR message happens around 500 times which has to be displayed over a 9600 baud connection :). Once this is done it will resume with:

** Phase 2 - Check Pathnames
ROOT INODE UNALLOCATED
ALLOCATE? yes

CG 0: BAD MAGIC NUMBER
CG 0: BAD MAGIC NUMBER
** Phase 3 - Check Connectivity
UNREF DIR I=1792 OWNER=0 MODE=40755
SIZE=512 MTIME=Nov 2 13:43 2012
RECONNECT? yes

NO lost+found DIRECTORY
CREATE? yes

CG 0: BAD MAGIC NUMBER
CG 0: BAD MAGIC NUMBER
DIR I=1792 CONNECTED. PARENT WAS I=2

** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
CG 0: BAD MAGIC NUMBER
FREE BLK COUNT(S) WRONG IN SUPERBLK
SALVAGE? yes

SUMMARY INFORMATION BAD
SALVAGE? yes

BLK(S) MISSING IN BIT MAPS
SALVAGE? yes

8 files, 8 used, 12430 free (14 frags, 1552 blocks, 0.1% fragmentation)

UPDATE CORRUPTED DUPLICATE SUPERBLOCKS? yes

UPDATE STANDARD SUPERBLOCK? yes

***** FILE SYSTEM WAS MODIFIED *****
Could not find any existing configuration.
Formatting target media /dev/da0 ...
Preparing to create slices on /dev/da0
/dev/da0: 2048000 sectors [C:1000 H:64 S:32 SS:512]
Shrinking slice 1 by 256 blocks for alignment
1+0 records in
1+0 records out
512 bytes transferred in 0.000671 secs (763143 bytes/sec)
Creating slices:
g c1000 h64 s32
p 1 0xA5 256 610048
p 2 0xA5 610304 610304
p 3 0xA5 1220608 763904
p 4 0xA5 1984512 63488
a 1
******* Working on device /dev/da0 *******
Computing layout of partitions in /dev/da0s1...
Shrinking partition a by 1792 blocks for alignment
Labeling /dev/da0s1:
bsdlabel: write to disk label supressed - label was as follows:
# /dev/da0s1:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
a: 608000 256 unused 0 0
c: 610048 0 unused 0 0 # "raw" part, don't edit
/dev/da0s1a: 296.9MB (607996 sectors) block size 16384, fragment size 2048
using 4 cylinder groups of 74.22MB, 4750 blks, 9600 inodes.
super-block backups (for fsck -b #) at:
32, 152032, 304032, 456032
Computing layout of partitions in /dev/da0s2...
Labeling /dev/da0s2:
bsdlabel: write to disk label supressed - label was as follows:
# /dev/da0s2:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
a: 610048 256 unused 0 0
c: 610304 0 unused 0 0 # "raw" part, don't edit
/dev/da0s2a: 297.9MB (610044 sectors) block size 16384, fragment size 2048
using 4 cylinder groups of 74.47MB, 4766 blks, 9600 inodes.
super-block backups (for fsck -b #) at:
32, 152544, 305056, 457568
Computing layout of partitions in /dev/da0s3...
Shrinking partition e by 256 blocks for alignment
Labeling /dev/da0s3:
bsdlabel: write to disk label supressed - label was as follows:
# /dev/da0s3:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
c: 763904 0 unused 0 0 # "raw" part, don't edit
e: 50944 256 unused 0 0
f: 712704 51200 unused 0 0
/dev/da0s3e: 24.9MB (50940 sectors) block size 16384, fragment size 2048
using 4 cylinder groups of 6.22MB, 398 blks, 896 inodes.
super-block backups (for fsck -b #) at:
32, 12768, 25504, 38240
/dev/da0s3f: 348.0MB (712700 sectors) block size 16384, fragment size 2048
using 4 cylinder groups of 87.00MB, 5568 blks, 11136 inodes.
super-block backups (for fsck -b #) at:
32, 178208, 356384, 534560
Computing layout of partitions in /dev/da0s4...
Shrinking partition e by 256 blocks for alignment
Labeling /dev/da0s4:
bsdlabel: write to disk label supressed - label was as follows:
# /dev/da0s4:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
a: 57344 6144 unused 0 0
c: 63488 0 unused 0 0 # "raw" part, don't edit
e: 5888 256 unused 0 0
/dev/da0s4a: 28.0MB (57340 sectors) block size 16384, fragment size 2048
using 4 cylinder groups of 7.00MB, 448 blks, 896 inodes.
super-block backups (for fsck -b #) at:
32, 14368, 28704, 43040
/dev/da0s4e: 2.9MB (5884 sectors) block size 16384, fragment size 2048
using 3 cylinder groups of 1.00MB, 64 blks, 128 inodes.
super-block backups (for fsck -b #) at:
32, 2080, 4128
Creating root filesystem layout ...
Downloading /junos-srxsme-12.1X46-D55.3-domestic.tgz from 192.168.0.75 ...
Time and ticks drifted too much, resetting synchronization...
Verified SHA1 checksum of /a/cf/install/junos-boot-srxsme-12.1X46-D55.3.tgz
Verified SHA1 checksum of /a/cf/install/junos-srxsme-12.1X46-D55.3-domestic
Creating var filesystem layout ...
Creating bsdlabel recovery information ...
Initializing alternate root ...
machdep.bootsuccess: 0 -> 1
machdep.nextbootdev: usb -> nand-flash
JUNOS requires BIOS version upgrade from 2.0 to 2.8
Upgrading to BIOS 2.8 ...
boot.upgrade.uboot="0xbfc00000"
boot.upgrade.loader="0xbfe00000"
Upgrading Loader...
#####################################
Verifying the loader image... OK
Upgrading U-Boot...
###############################################################################
Verifying the new U-Boot image... OK
WARNING: The new boot firmware will take effect when the system is rebooted.
Installation completed successfully, rebooting ...
Waiting (max 60 seconds) for system process `vnlru_mem' to stop...done
Waiting (max 60 seconds) for system process `vnlru' to stop...done
Waiting (max 60 seconds) for system process `bufdaemon' to stop...done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...0 done

syncing disks... All buffers synced.
Uptime: 9m9s
Rebooting...
cpu_reset: Stopping other CPUs

This was the first phase. The device will now reboot and start booting the fresh installed image and prepare itself for use:
U-Boot 1.1.6-JNPR-2.8 (Build time: Feb 10 2015 - 01:03:41)

Initializing memory this may take some time...
Measured DDR clock 266.62 MHz
SRX_100_HIGHMEM board revision major:0, minor:0, serial #: AT0812AF0793
OCTEON CN5020-SCP pass 1.1, Core clock: 500 MHz, DDR clock: 266 MHz (532 Mhz data rate)
DRAM: 1024 MB
Starting Memory POST...
Checking datalines... OK
Checking address lines... OK
Checking 512K memory for U-Boot... OK.
Running U-Boot CRC Test... OK.
Flash: 4 MB
USB: scanning bus for devices... 3 USB Device(s) found
scanning bus for storage devices... 1 Storage Device(s) found
Clearing DRAM........ done
BIST check passed.
Boot Media: nand-flash usb
Net: pic init done (err = 0)octeth0
POST Passed
Press SPACE to abort autoboot in 1 seconds
ELF file is 32 bit
Loading .text @ 0x8f0000a0 (246560 bytes)
Loading .rodata @ 0x8f03c3c0 (14144 bytes)
Loading .reginfo @ 0x8f03fb00 (24 bytes)
Loading .rodata.str1.4 @ 0x8f03fb18 (16516 bytes)
Loading set_Xcommand_set @ 0x8f043b9c (96 bytes)
Loading .rodata.cst4 @ 0x8f043bfc (20 bytes)
Loading .data @ 0x8f044000 (5760 bytes)
Loading .data.rel.ro @ 0x8f045680 (120 bytes)
Loading .data.rel @ 0x8f0456f8 (136 bytes)
Clearing .bss @ 0x8f045780 (11600 bytes)
## Starting application at 0x8f0000a0 ...
Consoles: U-Boot console
Found compatible API, ver. 2.8

FreeBSD/MIPS U-Boot bootstrap loader, Revision 2.8
(slt-builder@svl-ssd-build-vm06.juniper.net, Tue Feb 10 00:32:30 PST 2015)
Memory: 1024MB
[0]Booting from nand-flash slice 1
Un-Protected 1 sectors
writing to flash...
Protected 1 sectors
Loading /boot/defaults/loader.conf
/kernel data=0xb16d5c+0x134b2c syms=[0x4+0x8bbd0+0x4+0xcadc3]

Hit [Enter] to boot immediately, or space bar for command prompt.
Booting [/kernel]...
Kernel entry at 0x801000e0 ...
init regular console
Primary ICache: Sets 64 Size 128 Asso 4
Primary DCache: Sets 1 Size 128 Asso 64
Secondary DCache: Sets 128 Size 128 Asso 8
GDB: debug ports: uart
GDB: current port: uart
KDB: debugger backends: ddb gdb
KDB: current backend: ddb
kld_map_v: 0x8ff80000, kld_map_p: 0x0
Copyright (c) 1996-2016, Juniper Networks, Inc.
All rights reserved.
Copyright (c) 1992-2006 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
JUNOS 12.1X46-D55.3 #0: 2016-07-08 18:46:54 UTC
builder@quoarth.juniper.net:/volume/build/junos/12.1/service/12.1X46-D55.3/obj-octeon/junos/bsd/kernels/JSRXNLE/kernel
JUNOS 12.1X46-D55.3 #0: 2016-07-08 18:46:54 UTC
builder@quoarth.juniper.net:/volume/build/junos/12.1/service/12.1X46-D55.3/obj-octeon/junos/bsd/kernels/JSRXNLE/kernel
real memory = 1073741824 (1024MB)
avail memory = 509661184 (486MB)
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
Security policy loaded: JUNOS MAC/pcap (mac_pcap)
Security policy loaded: JUNOS MAC/runasnonroot (mac_runasnonroot)
netisr_init: !debug_mpsafenet, forcing maxthreads from 2 to 1
cpu0 on motherboard
: CAVIUM's OCTEON 5020 CPU Rev. 0.1 with no FPU implemented
L1 Cache: I size 32kb(128 line), D size 8kb(128 line), sixty four way.
L2 Cache: Size 128kb, 8 way
obio0 on motherboard
uart0: on obio0
uart0: console (9600,n,8,1)
twsi0 on obio0
dwc0: on obio0
usb0: on dwc0
usb0: USB revision 2.0
uhub0: vendor 0x0000 DWC OTG root hub, class 9/0, rev 2.00/1.00, addr 1
uhub0: 1 port with 1 removable, self powered
uhub1: vendor 0x0409 product 0x005a, class 9/0, rev 2.00/1.00, addr 2
uhub1: single transaction translator
uhub1: 2 ports with 1 removable, self powered
umass0: STMicroelectronics ST72682 High Speed Mode, rev 2.00/2.10, addr 3
cpld0 on obio0
pcib0: on obio0
Disabling Octeon big bar support
PCI Status: PCI 32-bit: 0xc041b
pcib0: Initialized controller
pci0: on pcib0
pci0: at device 2.0 (no driver attached)
pci0: at device 2.1 (no driver attached)
pci0: at device 2.2 (no driver attached)
gblmem0 on obio0
octpkt0: on obio0
cfi0: on obio0
Timecounter "mips" frequency 500000000 Hz quality 0
###PCB Group initialized for udppcbgroup
###PCB Group initialized for tcppcbgroup
da0 at umass-sim0 bus 0 target 0 lun 0
da0: Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 1000MB (2048000 512 byte sectors: 64H 32S/T 1000C)
Trying to mount root from ufs:/dev/da0s1a
MFSINIT: Initialising MFSROOT
Process-1 beginning MFSROOT initialization...
Creating MFSROOT...
/dev/md0: 20.0MB (40956 sectors) block size 16384, fragment size 2048
using 4 cylinder groups of 5.00MB, 320 blks, 640 inodes.
super-block backups (for fsck -b #) at:
32, 10272, 20512, 30752
Populating MFSROOT...
Creating symlinks...
Setting up mounts...
Continuing boot from MFSROOT...
Attaching /cf/packages/junos via /dev/mdctl...
Mounted junos package on /dev/md1...
S
chflags: /var/packages/*: No such file or directory
Media check on da0
Automatic reboot in progress...
** /dev/da0s1a (NO WRITE)
** Last Mounted on /
** Root file system
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
FREE BLK COUNT(S) WRONG IN SUPERBLK
SALVAGE? no

SUMMARY INFORMATION BAD
SALVAGE? no

BLK(S) MISSING IN BIT MAPS
SALVAGE? no

161 files, 75562 used, 73964 free (44 frags, 9240 blocks, 0.0% fragmentation)
mount reload of '/' failed: Operation not supported

Verified junos signed by PackageProductionEc_2016 method ECDSA
Verified jboot signed by PackageProductionEc_2016 method ECDSA
Verified junos-12.1X46-D55.3-domestic signed by PackageProductionEc_2016 method ECDSA
Checking integrity of BSD labels:
s1: Passed
s2: Passed
s3: Passed
s4: Passed
** /dev/bo0s3e
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 12436 free (28 frags, 1551 blocks, 0.2% fragmentation)
** /dev/bo0s3f
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 175282 free (34 frags, 21906 blocks, 0.0% fragmentation)
Checking integrity of licenses:
Checking integrity of configuration:
rescue.conf.gz: No recovery data
Loading configuration ...
Time and ticks drifted too much, resetting synchronization...
mgd: error: Cannot open configuration file: /config/juniper.conf
mgd: warning: activating factory configuration
Interface control process: [edit interfaces fe-0/0/1 unit 0]
Interface control process: 'family'
Interface control process: Ethernet-switching family not supported in HA mode for srx100h platform
mgd: error: configuration check-out failed
Warning: Commit failed, activating partial configuration.
Warning: Edit the router configuration to fix these errors.
Setting initial options: .
Starting optional daemons: usbd.
Doing initial network setup:
.
Initial interface configuration:
additional daemons: eventd.
Generating RSA key /etc/ssh/ssh_host_key
Generating public/private rsa1 key pair.
Your identification has been saved in /etc/ssh/ssh_host_key.
Your public key has been saved in /etc/ssh/ssh_host_key.pub.
The key fingerprint is:
e3:ba:b3:06:65:26:4f:5b:a9:92:02:c5:5c:99:46:e9 root@
The key's randomart image is:
+--[RSA1 2048]----+
| o oo+ |
| + = |
| . o . |
|. E = o |
| . O +S |
| . + +. . |
| . o . |
| o. |
| .++ |
+-----------------+
Generating DSA key /etc/ssh/ssh_host_dsa_key
Generating public/private dsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_dsa_key.
Your public key has been saved in /etc/ssh/ssh_host_dsa_key.pub.
The key fingerprint is:
47:85:2c:5b:fa:a1:b2:bd:ba:fe:ef:51:2d:7b:5d:22 root@
The key's randomart image is:
+--[ DSA 1024]----+
| . .. |
| . +. |
| =. |
| o.. . |
| So..E o .|
| . .... + o.|
| + . . . .|
| . . . . |
| .++o+o |
+-----------------+
Generating RSA2 key /etc/ssh/ssh_host_rsa_key
Generating public/private rsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_rsa_key.
Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub.
The key fingerprint is:
40:7a:16:30:b7:a0:ed:c0:2c:6c:2d:be:5e:00:f0:28 root@
The key's randomart image is:
+--[ RSA 2048]----+
|. +.+ |
|o=.o * o |
|E+*.o = |
|=..o o . |
| o . S |
| o |
| . . |
|. . |
| . |
+-----------------+
Generating ECDSA key /etc/ssh/ssh_host_ecdsa_key
Generating public/private ecdsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_ecdsa_key.
Your public key has been saved in /etc/ssh/ssh_host_ecdsa_key.pub.
The key fingerprint is:
59:8f:da:93:f7:8b:0c:0f:2c:1c:b7:fb:7d:ba:91:d4 root@
The key's randomart image is:
+--[ECDSA 256]---+
| |
| |
| . |
| o o . |
| S o . . E|
| . * o . . |
| + O . o |
| . O + ..|
| ..= *= |
+-----------------+
Generating ED25519 key /etc/ssh/ssh_host_ed25519_key
Generating public/private ed25519 key pair.
Your identification has been saved in /etc/ssh/ssh_host_ed25519_key.
Your public key has been saved in /etc/ssh/ssh_host_ed25519_key.pub.
The key fingerprint is:
75:52:bb:29:4d:06:6d:b9:e4:ce:6f:18:e2:d8:ce:7c root@
The key's randomart image is:
+--[ED25519 256--+
| .... |
| o=. |
| o+=. |
| . *oo |
| S .o+ |
| ..+ |
| + . + |
| .oo E o |
| .+. . |
+-----------------+
Ignoring watchdog timeout during boot/reboot
Additional routing options:kern.module_path: /boot//kernel;/boot/modules -> /boot/modules;/modules/ifpfe_drv;/modules;
kld netpfe drv: ifpfed_dialer ipsec kld.
Doing additional network setup:.
Starting final network daemons:.
setting ldconfig path: /usr/lib /opt/lib
starting standard daemons: cron.
Initial rc.mips initialization:.
Local package initialization:.
starting local daemons:set cores for group access
.
kern.securelevel: -1 -> 1
Creating JAIL MFS partition...
JAIL MFS partition created
boot.upgrade.uboot="0xBFC00000"
boot.upgrade.loader="0xBFE00000"
JUNOS requires backup BIOS version upgrade from 2.0 to 2.8
Upgrading to BIOS 2.8 ...
Upgrading Secondary U-Boot...
###############################################################################
Verifying the new U-Boot image... OK
Boot media /dev/da0 has dual root support
** /dev/da0s2a
FILE SYSTEM CLEAN; SKIPPING CHECKS
clean, 74476 free (28 frags, 9306 blocks, 0.0% fragmentation)
Fri Sep 9 18:12:31 UTC 2016

Amnesiac (ttyu0)

login:

Continue configuration:
The default account is “root” with no password. If you have no experience with the setup of such device purely via CLI I advise you to hook up a LAN cable between your machine and the first ethernet port of the device (that is port 0/0) and set up a static IP in the 192.168.1.0/24 range. You should then be able to browse to the URL:
http://192.168.1.1/

The welcome screen should be like below:
j-web

You can login with “root” and will be welcomed by a setup wizard which will guide you through the rest of the setup based on several simple questions.

In Software

Updating the firmware on a Juniper SSG5 via CLI

 - 31-08-2016

SSG5_Recently I got an old Juniper SSG5 router which was taken out of service. The unit itself dates back from 2010 and never received updates while it was operational. Before taking it into production again I want to make sure that the firmware is up-to-date.

Prerequisites:
Before we can start updating the unit we need to download the firmware. Since the device is EOL we can download the firmware free of charge (but requires you to register once). You can find the download page here.

Although not required, a TFTP server is needed to save the initial configuration and load the new firmware from. In this guide I assume that a TFTP server is available in the network. In this guide my TFTP server listens on IP-address 192.168.0.75.

If you are going to upgrade from a firmware earlier than the 6.3 release you need to download a new key file as well which holds the MD5 checksums for the new images.

Factory resetting the switch:
As this unit came out of production it holds a certain configuration and password I don’t know. So in order to use the device we need to reset it to the factory defaults first. With a paperclip you can reset the device (reset button is on the back) as follows:

1: While the device is running press and hold the reset button for ~6 seconds and release
2: Wait ~2 seconds
3: Press the reset button again for around ~6 seconds and release

The unit should now reboot and restore the factory image into flash. This may take up to 5 minutes to finish.

If you already have this unit in production do not perform these steps as it will wipe your entire configuration!

Logging in on the switch:
The switch has a serial console port which is the most convenient way to administer it. It is also possible to use telnet and/or SSH to log in but it has to be enabled manually per port on the device and is by default disabled.

The console settings are 9600baud 8N1 with no flow control.

The default username and password are both “netscreen“. When logged in successfully you will see the following prompt:
login: netscreen
password:
ssg5-serial->

Checking the current software version:
When logged in to the switch perform the following command to see the current version:

ssg5-serial-> get system
Product Name: SSG5-Serial
Serial Number: 0162122010003603, Control Number: 00000000
Hardware Version: 0710(0)-(00), FPGA checksum: 00000000, VLAN1 IP (0.0.0.0)
Flash Type: Samsung
Software Version: 6.2.0r5.0, Type: Firewall+VPN
Feature: AV-K
Compiled by build_master at: Thu Jan 28 11:42:26 PST 2010
Base Mac: 28c0.dae8.e140
File Name: ssg5ssg20.6.2.0r5.0, Checksum: 23364b4e
, Total Memory: 256MB

There is more output to be shown, you can press CONTROL + C to cancel the listing.

Saving the configuration:
This step is only needed when you already have the device in production. If you have just factory reset the device this step can be skipped.

When logged in to the console enter the following to save the configuration:
save config to tftp 192.168.0.75 ssg5_31-8-2016.cfg

This will save the current configuration with filename “ssg5_31-8-2016.cfg” to the TFTP server running on 192.168.0.75. If this goes well you should see:
Read the current config.
Save configurations (4935 bytes) to ssg5_31-8-2016.cfg on TFTP server 192.168.0.75.
!!!!!!!!!!!!!!!!!!!!!!
tftp transferred records = 10
tftp success!

TFTP Succeeded
ssg5-serial->

Preparing the firware:
The latest firmware for the SSG5 as of writing is version 6.3.0r22. You should have a file called “ssg5ssg20.6.3.0r22.0.zip”. Unzip the contents of the zip into your TFTP directory (usually /srv/tftp/).

Optionally you may have a bootloader update as well as I had to update as well. You will also have a file called “Loadssg5ssg20v132.d.zip” which needs to be unzipped in the TFTP directory as well.

Updating the firmware:
If you want to update the firmware and are coming from a release before 6.3 you need to update the image MD5 keys first. Make sure you have extracted the zip file containing the keys in your TFTP directory. After unzipping you will have a “imagekey.cer” file that we need to flash first.

You may want to check the keys installed first. you can do this with:
ssg5-serial-> exec pki test skey
exec pki test .
Flash base = 0x51000000, Flash end = 0x0, sector size= 0x4000

KEY1 N/A len =432
308201ac02010002818100fd7f53811d75122952df4a9c2eece4e7f611b7523cef4400c31e3f800

KEY2 N/A len =432
308201ac02010002818100fd7f53811d75122952df4a9c2eece4e7f611b7523cef4400c31e3f800

KEY3 N/A len =432
308201ac02010002818100fd7f53811d75122952df4a9c2eece4e7f611b7523cef4400c31e3f800

You see that the string starts with 308201ac. It’s eigth character is a c which indicates that you have a firmware prior to 6.3 running and you have to update the keys first. Firmware releases in the 6.3 branch have the c replaced with a d.

Issue the following command to load the new keys:
ssg5-serial-> save image-key tftp 192.168.0.75 imagekey.cer
Load file from TFTP 192.168.0.75 (file: imagekey.cer).
!!!!!
tftp received octets = 863
tftp success!
Done

TFTP Succeeded
ssg5-serial->

Now we can check the keys again and you should notice that the “c” is changed to a “d”:
ssg5-serial-> exec pki test skey
exec pki test .
Flash base = 0x51000000, Flash end = 0x0, sector size= 0x4000

KEY1 N/A len =433
308201ad02010002818100fd7f53811d75122952df4a9c2eece4e7f611b7523cef4400c31e3f800

KEY2 N/A len =433
308201ad02010002818100fd7f53811d75122952df4a9c2eece4e7f611b7523cef4400c31e3f800

KEY3 N/A len =433
308201ad02010002818100fd7f53811d75122952df4a9c2eece4e7f611b7523cef4400c31e3f800

Now that the keys are up-to-date we can start the actual firmware update:
ssg5-serial-> save software from tftp 192.168.0.75 ssg5ssg20.6.3.0r22.0 to flash
Load software from TFTP 192.168.0.75 (file: ssg5ssg20.6.3.0r22.0).
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
tftp received octets = 13381258
tftp success!

TFTP Succeeded
Save to flash. It may take a few minutes ...platform = 25, cpu = 12, version = 18
update new flash image (02a676a0,13381258)
platform = 25, cpu = 12, version = 18
offset = 20, address = 5800000, size = 13381180
date = 2669, sw_version = 31808000, cksum = 59d01749
Image authenticated!
Program flash (13381258 bytes) ...
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++e
Done

This step takes up to 5 minutes to download and install the firmware.

Now that the new firmware is installed we need to reboot the device to load the new firmware. This can simply be done with the reset command:
ssg5-serial-> reset
System reset, are you sure? y/[n] y
In reset ...

After around 5 minutes the device should be back online again and we want to check if the new version is active. Log in and retrieve the system information:
ssg5-serial-> get system
Product Name: SSG5-Serial
Serial Number: 0162122010003603, Control Number: 00000000
Hardware Version: 0710(0)-(00), FPGA checksum: 00000000, VLAN1 IP (0.0.0.0)
Flash Type: Samsung
Software Version: 6.3.0r22.0, Type: Firewall+VPN
Feature: AV-K
BOOT Loader Version: 1.3.2
Compiled by build_master at: Wed Mar 9 07:57:20 PST 2016
Base Mac: 28c0.dae8.e140
File Name: ssg5ssg20.6.3.0r22.0, Checksum: 11a822d0
, Total Memory: 256MB

In this case the upgrade went with success!

Updating the bootloader firmware:
Updating the bootloader cannot be performed when the OS is running and can only be performed from the bootloader itself. Reset the device and halt the boot process to gain access to the bootloader prompt:

ssg5-serial-> reset
System reset, are you sure? y/[n] y
In reset ...

Juniper Networks SSG5 Boot Loader Version 1.3.2 (Checksum: A1EAB858)
Copyright (c) 1997-2006 Juniper Networks, Inc.

Total physical memory: 256MB
Test - Pass
Initialization - Done

Hit any key to run loader
Hit any key to run loader

Serial Number [0162122010003603]: READ ONLY
HW Version Number [0710]: READ ONLY
Self MAC Address [28c0-dae8-e140]: READ ONLY
Boot File Name [ssg5ssg20.6.3.0r22.0]: Loadssg5ssg20v132.d
Self IP Address [192.168.2.26]: 192.168.0.7
TFTP IP Address [192.168.2.100]: 192.168.0.75

Save loader config (56 bytes)... Done

Loading file "Loadssg5ssg20v132.d"...
rtatatatatatatatatatatatatatatatatatatatatatatatatatat

Loaded Successfully! (size = 407,771 bytes)

Image authenticated!

Save to on-board flash disk? (y/[n]/m)

Enter “y” at the above question to flash the bootloader firmware.

Save to on-board flash disk? (y/[n]/m) Yes!

Saving system image to on-board flash disk...
Done! (size = 407,771 bytes)

Run downloaded system image? ([y]/n)

Enter “y” at the above question to run the bootloader update:

Run downloaded system image? ([y]/n) Yes!
Check on-board Boot Loader... Update needed!

Are you sure you want to update Boot Loader? (y/n)

Enter “y” to finalize the bootloader update:

Read product information of on-board boot flash device:
Manufacturer ID = 1f
Device ID = 13
Additional Device ID = 10

Boot flash device is AT49LV040B

Erase on-board boot flash device.......... Done

Update Boot Loader....................................................

Verify Boot Loader... Done

Boot Loader has been updated successfully!

Please hit any key to reboot the system...

Press any key to reboot the system. The bootloader update has finished! After this the bootloader tries to update at every reboot which is very inconvenient. You can solve this by removing the bootloader update file from the TFTP folder.

In Software

Testing your internet connection speed from the commandline

 - 11-08-2016

speedometer
Sometimes you may want to check the internet connection of the provider you are connected to. Although there are various versions that are browser related I rather want to have a method like this commandline based.

There is always the method of downloading a large file off the internet to see how fast it performs, but it’s not very convenient. Upon some searching I came across the tool “speedtest-cli” which is able to perform such speedtests directly from the commandline.

Requirements:
Speedtest-cli is a python program and can be installed using pip. If you don’t have pip installed you can install it with the commands below:

Debian/Ubuntu:
apt-get install python-pip

CentOS/RedHat:
yum install python-pip

ArchLinux:
pacman -Sy python-pip

Installing speedtest-cli:
When pip is installed, run the following to install it:
pip install speedtest-cli

The output will be like:
Collecting speedtest-cli
Using cached speedtest_cli-0.3.4-py2.py3-none-any.whl
Installing collected packages: speedtest-cli
Successfully installed speedtest-cli-0.3.4

Using speedtest-cli
Now that it is installed, you can simply start it by running:
speedtest-cli

It will automatically determine the nearest mirror and start the test. Output will be like it is below (IP masked):
[fileserver ~]# speedtest-cli
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from Ziggo (1.2.3.4)...
Selecting best server based on latency...
b'Hosted by INTERACTIVE 3D B.V. (Rotterdam) [14.50 km]: 16.043 ms'
Testing download speed........................................
Download: 236.53 Mbit/s
Testing upload speed..................................................
Upload: 27.37 Mbit/s

I have a 300/30Mbit internet connection at home and had some downloads ongoing at that time, so the results are what I’ve expected from it.

In Server, Software

Install Observium on ArchLinux

 - 09-08-2016

Hamster-transparentRecently I’ve obtained an old 8-port switch from the office (a WWP LightningEdge 310). That switch has a management port for managing the switch via cli but also is capable of sending SNMP data so we can monitor the status of the switch in tools like Cacti or Observium. I’ve tried Cacti before and although it’s very mature and has a lot of functions it’s also a big puzzle to find out how it works with all of it’s templates and graphs. I’ve also tried Observium and this does the job perfectly. Since the installation of Observium is well documented for RedHat and Debian like systems and I only have ArchLinux based systems I’ve tried installing it on Arch, this post will therefore describe the process of installing and running Observium on a ArchLinux install.

Why Observium?
Observium is able to poll SNMP data like Cacti but it’s much more convenient in what it is able to show and graph. By default Cacti has some basic templates for traffic monitoring but it’s not very thorough. Observium is able to graph all SNMP data that is available for processing and does the work almost completely for you. In my case with the LE-310 switch it is able to show all interface statistics, optic status, system status etcetera as where Cacti was able to only monitor the first.

System requirements:
Observium has pretty high system requirements. Since I’m only monitoring 1 switch in this install I can imagine that the actual requirements would be much lower. My testing case was done on a very old HP machine (s7720.nl) with the following specs:

CPU: AMD X2 3800+ 2GHz dualcore
RAM: 4GB DDR2 800 ECC
LAN: Realtek 8101CP 100Mbit
HDD: 1TB Seagate Constellation 7200RPM

As said, the machine is very old (almost 9 years). As it’s only used for testing purposes I found the machine handling Observium pretty well. The web pages are generated pretty fast with PHP7 and no Opcode caching. The only thing that was notably slower was the generation of the graphs which in this case is the CPU intensive part. It could take up to several seconds to load ~15 – ~20 graphs on a page, still okay for me. Should you want more speed you might want to consider a more modern machine with a newer CPU for processing the graph data.

Installing required software:
Before we can install Observium we need to install some packages that consist of a LAMP stack, some PHP modules and software that is needed so Observium can do it’s work properly. From the commandline/terminal elevate to root and install the required software:

pacman -Sy apache mariadb php php-apache php-mcrypt php-gd php-snmp net-snmp fping rrdtool whois mtr ipmitool graphviz imagemagick python-pip nmap

For the cron scripts to work we need to have pymysql installed as well:
pip3 install pymysql

Then go back to your normal user account and install the php-pear package, I’ve done this like (as root):
su jeffrey
yaourt- Sy php-pear
exit

Configuring the installed software:
By default the Apache and MariaDB services are not started and have to be “enabled” so they start at boot and have to be “started” in order for them to work.

Enable the apache and MariaDB services:
systemctl enable httpd.service
systemctl enable mariadb.service

If you have just installed MariaDB and haven’t set it up before it’s best to do now to finish and secure your installation. To finish the installation run:
mysql_install_db --user=mysql --basedir=/usr --datadir=/var/lib/mysql

This will take a couple of seconds to finish and results in a working MariaDB server setup but with no password for the root account, start MariaDB and set it via:
systemctl start mariadb.service
mysql_secure_installation

The script makes most of the default choices for you, it’s best to keep them as suggested!

We haven’t started Apache yet as we have to change the default config as well and saves us a Apache restart. Open the main configuration file:
vim /etc/httpd/conf/httpd.conf

Locate the following line:
#LoadModule rewrite_module modules/mod_rewrite.so

And replace it with:
LoadModule rewrite_module modules/mod_rewrite.so

Next locate the following line:
LoadModule mpm_event_module modules/mod_mpm_event.so

Replace it with:
#LoadModule mpm_event_module modules/mod_mpm_event.so

And change the line underneath it from:
#LoadModule mpm_prefork_module modules/mod_mpm_prefork.so

To:
LoadModule mpm_prefork_module modules/mod_mpm_prefork.so

In that same LoadModule list add to the end of the list the following line so that PHP scripts work as well:
LoadModule php7_module modules/libphp7.so

Search in the file for Include and insert the following line underneath the last Include:
Include conf/extra/php7_module.conf

Later in the file there is a DocumentRoot section, within the Directory directive, replace everything with:
DirectoryIndex index.php
Options Indexes FollowSymLinks MultiViews
AllowOverride All
Require all granted

Change the DocumentRoot and Directory itself as well from:
DocumentRoot "/srv/http"

To:
DocumentRoot "/srv/http/html"

Save the changes and exit the file.

Now we have to change some PHP settings as well, open the following file:
vim /etc/php/php.ini

Search for the following line:
extension_dir = "/usr/lib/php/modules/"

And add the following lines beneath it:
extension=snmp.so
extension=sockets.so
extension=mysqli.so
extension=pdo_mysql.so
extension=gd.so
extension=mcrypt.so

Save the changes and exit the file. Now let’s start Apache as well:
systemctl start httpd.service

Downloading Observium:
By default the docroot of Apache on ArchLinux is in /srv/http/. One thing that you must know is that Observium must run from within the docroot itself and will not run using a alias or subdirectory. This is because it’s programmed to always run from the docroot (and that is bad).

So first we navigate to the docroot directory:
cd /srv/http/

Download the latest Observium archive:
wget http://www.observium.org/observium-community-latest.tar.gz

And unpack it straight into the current docroot directory:
tar --strip-components=1 -vxzf observium-community-latest.tar.gz

Remove the downloaded installation archive:
rm -f observium-community-latest.tar.gz

Configuring Observium:
In order to use Observium we need to create a configfile with the database credentials and set up a database. First we will setup the database, so log in to MySQL as the root user with the password you’ve set earlier:
mysql -u root -p

Create a database, new database user and password:
mysql> CREATE DATABASE observium DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
mysql> GRANT ALL PRIVILEGES ON observium.* TO 'observium'@'localhost'
-> IDENTIFIED BY '';

Exit MySQL and setup the config file:
cp config.php.default config.php
vim config.php

You have to change the following lines to what you have setup above with the MySQL database setup:
// Database config --- This MUST be configured
$config['db_extension'] = 'mysqli';
$config['db_host'] = 'localhost';
$config['db_user'] = 'USERNAME';
$config['db_pass'] = 'PASSWORD';
$config['db_name'] = 'observium';

You also need to change the Observium root directory from:
#$config['install_dir'] = "/opt/observium";

To:
$config['install_dir'] = "/srv/http";

And save your changes. Now we are going to install the database schema, run the following:
./discovery.php -u

This will import the basic database schema into the Observium database. It may take up to a couple of minutes depending on the speed of the system, mostly disk related. The output is as shown below:
Install initial database schema ... done.
-- Updating database/file schema
252 -> 253 ... (db) done.
253 -> 254 ... (db) done.
254 -> 255 ... (db) done.
255 -> 256 ... (php)
256 -> 257 ... (php)
257 -> 258 ... (php)
258 -> 259 ... (db) done.
259 -> 260 ... (php)
260 -> 261 ... (db) done.
261 -> 262 ... (php)
262 -> 263 ... (db) done.
263 -> 264 ... (db) done.
264 -> 265 ... (db) done.
265 -> 266 ... (db) done.
-- Done.

We also need to create logging and RRD data directories:
mkdir /srv/http/rrd
mkdir /srv/http/logs
chmod 777 /srv/http/{rrd,logs}

The last step is to fix a issue with PEAR. Observium uses PEAR for several additional PHP modules, but looks for the PEAR.php file in it’s own pear directory which isn’t there. If you have the php-pear package installed before from yaourt you can solve this issue by copying the PEAR.php file from the pear directory over to the pear directory from Observium:
cp /usr/share/pear/PEAR.php /srv/http/libs/pear/

If you don’t copy the PEAR.php file you will not be able to add devices to your Observium install!

Creating a new user for Observium:
Before we can log in to the web interface of Observium we need to create a user for this purpose:
cd /srv/http/

Run the following command to create a user called “jeffrey” with a password “password” and admin rights “10”:
./adduser.php jeffrey password 10

Which should result in:
Observium CE 0.16.1.7533
Add User

User jeffrey added successfully.

Logging in to the web interface of Observium:
In your browser you can navigate to the IP-address or hostname, this should result in the login screen of Observium like shown below:
Observium

When logged in you should be able to add hosts and devices. Do note that they cannot be added by IP-address and always have to be entered as hostnames. Should this be an issue you can use the local /etc/hosts file on your Observium machine of fix the hostnames your DHCP server hands out.

Automatic polling for Observium:
It’s possible to poll the data for Obserium automatically. For this we will use the cron mechanism from the system. As root, edit the current cron:
crontab -e

And add the following 3 lines:
33 */6 * * * /srv/http/discovery.php -h all > /dev/null 2>&1
*/5 * * * * /srv/http/discovery.php -h new > /dev/null 2>&1
*/5 * * * * /srv/http/poller-wrapper.py 2 > /dev/null 2>&1

Save the cron and from now on every 5 minutes the new SNMP data will be polled for the devices you have configured!

In Server

Fix ProFTPd [unable to lstat AuthUserFile] error on DirectAdmin

 - 22-06-2016

directadmin
Recently I’ve updated the ProFTPd software on one of the servers at work and ran into the following issue after custombuild finished the update:
SNIP:
Restarting ProFTPd.
Shutting down proftpd: [ OK ]
Starting proftpd: 2016-06-10 10:12:35,133 server.example.net proftpd[3367]: mod_auth_file/1.0: unable to lstat AuthUserFile '/usr/local/directadmin/data/users/morpheus/ftp.passwd': No such file or directory
2016-06-10 10:12:35,134 server.example.net proftpd[3367]: fatal: AuthUserFile: unable to use /usr/local/directadmin/data/users/morpheus/ftp.passwd: No such file or directory on line 4 of '/etc/proftpd.vhosts.conf'
[FAILED]

The issue here is that the ProFTPd server no longer accepts the vhost config file but still parses it (hence the error above). So the real problem is that there is a user “morpheus” on the server (in my case) which has it’s own vhost config in the ProFTPd service but for that user the password file was not present anymore.

Upon looking into the vhost configuration file of ProFTPd learned me that there was indeed a manual created vhost section for the user “morpheus”:

bash-3.00# cat /etc/proftpd.vhosts.conf
[VirtualHost 194.60.207.182]
ServerName "ProFTPd"
ExtendedLog /var/log/proftpd/194.60.207.182.bytes WRITE,READ userlog
AuthUserFile /usr/local/directadmin/data/users/morpheus/ftp.passwd
[/VirtualHost]

This itself isn’t an issue as long as the ftp.passwd file for the user exists. When examining this further I found out that the user “morpheus” didn’t exist on the server anymore! So here it was actually pretty clear why the error showed up as the given ftp.passwd didn’t exist anymore because the user was removed from the system in the past but the vhost config was not properly cleaned.

To solve it in one go we can simply move away the vhost configuration file of ProFTPd to a safe location (as it’s deprecated now anyway):
bash-3.00# mv /etc/proftpd.vhosts.conf /root/

After moving the file out of the way start ProFTPd again and you should see no errors and the service should be reachable again:
bash-3.00# /etc/init.d/proftpd restart
Shutting down proftpd: [FAILED]
Starting proftpd: [ OK ]

In DirectAdmin, Server