EPICS for the Bewildered

The IOC (VME running vxWorks)

First and foremost, you'll need to get an IOC which can talk to your thingamajig (or, more likely, a computer which can talk to the card which talks to your thingamajig). As of EPICS 3.14.3, the computer you can use as your IOC is pretty wide open and is more a function of OS than hardware. We use VME crates (running vxWorks on a Motorola processor) here at the beam-line, but you can just as easily use a Linux box or even a Mac.

• Assembling the VME crate

To get started, you need:

Once these are assembled according to Motorola's instructions (TRIPLE CHECK the connections to the transition module if you're using it), you need to connect the serial port (COM1 on the 761 transition board (DB9M) and DEBUG on the MVME5100) to your PC so that you can talk to the controller using something like minicom. For the 761 transition board I just used a stadard female-female DB9 cable. However, the RJ-45 serial connector for the MVME5100 is non-standard and has the following (null modem) pin-outs:

RJ-45 1234 5678
DB9 1852 3N/C76
The connection was 9600 8N1.

Once you have successfully connected to the controller, on power-up, you should see this:

For the 2700...
Copyright Motorola Inc. 1988 - 1998, All Rights Reserved                       

PPC1 Debugger/Diagnostics Release Version 4.2 - 11/12/98 RM01                
COLD Start
Local Memory Found =08000000 (&134217728)

MPU Clock Speed =233Mhz                                                        

BUS Clock Speed =67Mhz                                                       

WARNING: Keyboard Not Connected                                            

Reset Vector Location  : ROM Bank A                                      
Mezzanine Configuration: Single-MPU                                     
Current 60X-Bus Master : MPU0                                          
Idle MPU(s)            : NONE                                         

System Memory: 128MB, ECC Enabled (ECC-Memory Detected)             
L2Cache:       1024KB, 117Mhz                                      

For the 5100...
Copyright Motorola Inc. 1988 - 2002, All Rights Reserved                       

PPC6 Debugger/Diagnostics Release Version 2.2 - 04/16/02 RM03                
COLD Start
Local Memory Found =20000000 (&536870912)

MPU Clock Speed =450Mhz                                                        

BUS Clock Speed =100Mhz                                                       

Reset Vector Location  : ROM Bank B                                      
Mezzanine Configuration: Single-MPU                                     
Current 60X-Bus Master : MPU0                                          
Idle MPU(s)            : NONE                                         

L2Cache:       1024KB, 180Mhz                                      
System Memory: 512MB, ECC Enabled (ECC-Memory Detected)             

The instructions for both the PPC-Bug and PPC-Diag commands are located on the Motorola Computer Group's Web Page.

• Installing vxWorks

At the moment, the controller card is just running a simple debugger. It's now time to install an operating system. Here at the beam-line, we seem to have a preference for vxWorks because we're government funded.

First and foremost, you will need two files: The vxWorks boot image (to be loaded into the controller card's flash memory) and the vxWorks kernel image. Both of these binary files are created under the target/config/<board> directory of the Tornado distribution.

(It may be necessary to first build the cross-compiler if you don't have the vxWorks binaries in hand. I already had the binaries, so I didn't have to do this, but you might want to skip ahead and do that before coming back here).

= Installing the vxWorks Boot Image

Once the bootrom and kernel image have been made, we have to get the vxWorks boot image onto a machine connected to the LAN. This will involve setting up a TFTP server somewhere on your network:

  1. Install the tftp-server rpm on the server machine.
  2. Create a world-readable directory for tftp (say, /var/tftp)
  3. In the file "/etc/xinetd.d/tftp" :
    1. Change "disable" to "no".
    2. Add an "only_from =" line to control access. (man xinetd.conf)
    3. Change the "server_args" -s argument to your tftp directory
  4. Make sure tftp is chkconfig'ed on.
  5. /etc/init.d/xinetd restart

Now follow the instructions for installing the VxWorks boot image for your controller card. Below, I mention some caveats for various boards.

The MVME2700 and MVME5100 have of soldered flash EPROM as well as 1MB of socketed Flash EPROM. [The MVME2700 has 4 MB of soldered flash and the MVME5100 uses four 4 MB soldered AMD-D323GT90V1 chips for bank "A" (for a total of 16 MB) and two 512 KB 32-pin AM29LV040B chips for bank "B"]

The goal is to load the vxWorks boot image into the soldered EPROM in bank "A" (faster CPU access) while operating under the PPCx-Bug image installed in the socketed flash in bank "B". Then we boot into the new OS on the soldered flash.

A board jumper determines which EPROM to boot from. For the MVME2700, it's J9 "(near the board's bottom handle)". For the MVME5100, it's J7 "(next to the screw holding down the heatsink)". Remove power from the crate. Set this jumper so that it connects pins 2 and 3. This selects the socketed EPROM bank "B." We now attempt to put the vxWorks boot image into the soldered flash in bank "A."

Connect the ethernet cable to the controller (or 761 transition board), and power up the system. You should see the following line in the boot sequence:

Reset Vector Location  : ROM Bank B 

Once you get to the "PPCx-Bug>" prompt, the first order of business is to set the system clock:

  set MMDDYYhhmm
I typically set GMT. You can query the time using the "time" command.

Now we need to tell the computer from where to download the image. This is done using the program "niot." The "Client IP Address" will be the IP address of the board and the "Server IP Address" is the IP address of the machine which has the image.

A response of "." to any question exits the program.

   PCx-Bug> niot
   Controller LUN =00?
   Device LUN     =00?
   Node Control Memory Address =xxxxxxxx?
   Client IP Address      = 164.54.204.xxx
   Server IP Address      = 164.54.204.yyy
   Subnet IP Address Mask = 
   Broadcast IP Address   =
   Gateway IP Address     = 164.54.204.___
   Boot File Name ("NULL" for None)     =? .

   Update Non-Volatile RAM (Y/N)? y

Let's test that setting with the "nping" command:

PPCx-Bug>nping 0 0 164.54.204.xxx 164.54.204.yyy 1
Which asks the source (164.54.204.x) to ping the destination (164.54.204.y) one time (the last parameter is in hexadecimal). You should get the response
Source IP Address      = 164.54.204.xxx
Destination IP Address = 164.54.204.yyy
Number of Packets Transmitted =1, Packets Lost =0, Packet Size =128  

N.B.: If there was recently another device on the server IP address (such as a board you just programmed), it may take some time for the ARP database to refresh on the gateway (on the order of 15 minutes). This can be remedied if you have root priveleges on the tftp server. First, find the controller card's Ethernet address by typing "cnfg". Then, on the server (as root), execute:
server# arp -s <client IP> <client ethernet address>
Don't forget to add the colons to the ethernet address (cnfg will just show a string of 12 hex characters).

Once this works, the image is downloaded (via tftp) into memory via the "niop" command. "The file name below is set to the location of the boot image in [the server's] TFTP area:"

PPCx-Bug> niop
   Controller LUN =00?
   Device LUN     =00?
   Get/Put        =G?
   File Name      =? vxworks/bootrom.mvXXXX
   Memory Address =00004000?
   Length         =00000000?
   Byte Offset    =00000000?
Notice that the image begins at address 4000.

"[Finally], the "pflash" command is used to put [the boot image] into the FLASH chips soldered onto the mezzanine board." The command is defined as

pflash <Source Base Address>:<Source End Address> <Destination Address>
For the MVME2700 the destination address is FF000100. However, for the MVME5100 the destination address depends on the size of the soldered flash.

64 KB of processor memory (beginning at FEF80000) is devoted to the system memory controller (SMC) registers. The "ROM A" base address is loated in the 12 bits beginning at FEF80050. Type:

PPC6-Bug> md fef80050
FEF80050  F40C0006 00000000 FF800006 00000000  ................
FEF80060  00000000 00000000 00000000 00000000  ................
So the ROM A flash addressing begins at F4000000 (the "C" in the lowest four bits of the second byte means that rom_a_64=1 (64 bits wide) and the size of ROM A is "100" which is code for 16 MB). We begin loading the flash memory 256 bits (32 bytes) in. In this case, that would be at address F4000100. So now type:
   PPCx-Bug> pflash 4000:FFF00 <DEST>
where <DEST> = FF000100 for an MVME2700 or F4000100 for the MVME5100 described above.
   Program FLASH Memory (Y/N)? y
   FLASH Memory Programming Complete                  
At this point, we can boot from the vxWorks image. Power down the crate and replace the ROM select jumper (J9 or J7) across pins 1 and 2. When power is re-applied, the vxWorks boot countdown should start. "You'll need to stop this before it finishes to set the vxWorks boot parameters...."

End of thievery, Andrew ;-)

= Configuring the Network (rsh issues)

At this point we need to delineate the host linux box. We have decided to use "rsh" which, as you know, is not terribly secure. The following circumstances seem to make RSH "ok" for now:

  1. We set the host:/etc/xinetd.d/rsh file to accept connections only inside the subnet (man xinetd.conf)
  2. No sensitive information will be transferred
  3. A .rhosts file in the IOC's home directory will keep passwords from being sent cleartext
  4. When testing rsh from another computer, make sure iptables is not blocking the return connection or you will see

    poll: protocol failure in circuit setup

Set up rsh and place the current vxWorks binary under host:/home/<username>/<subdir>/

= Configuring the vxWorks Boot Program

Now that the vxWorks boot executable has been loaded into flash memory, we need to configure it to download the actual vxWorks OS. This is done by halting the auto-boot and typing "c" at the "[VxWorks Boot]:" prompt. The following is a description of our configuration:

[VxWorks Boot]: c

'.' = clear field;  '-' = go to previous field;  ^D = quit

 boot device          : dc0 / fei0
 processor number     : 0
 host name            : iochost
 file name            : /home/username/crate3/mv2700-asd4
 inet on ethernet (e) : 164.54.204.xx:aabbccdd
 inet on backplane (b):
 host inet (h)        : 164.54.204.yy
 gateway inet (g)     :
 user (u)             : username
 ftp password (pw) (blank = use rsh):
 flags (f)            : 0x0
 target name (tn)     : crate3
 startup script (s)   : /home/username/crate3/st.cmd 
 other (o)            :     

Once this is set up, type "@" and hope the VME crate can load the files it needs from the host. If it works, you should get a vxWorks prompt and you can start playing with the vxWorks OS.

Useful vxWorks Commands
version Prints out the vxWorks version information and boot line
i Equivalent to linux "ps"
< Imports a text file of commands (e.g. < my_script )
ld < Imports compiled code (e.g, ld < iocCore )
d 0xaddress Dump memory starting at hex location address

In some cases, "d" will fail. Here is one reason from the documention of a card:

    "NOTE: the VSAM data must be accessed as A24/D32. If you attempt
     to access the VSAM data from the vxWorks shell, using the "d"
     command, you will get an access violation. This is because "d"
     accesses the data 1-byte at a time."
vxMemProbe(0xaddress, RW, BYTES, LVAR) Will probe an address and trap bus errors (exit -1 if bus error)
                ioc> x = 1
                ioc> vxMemProbe (0xaddress, 0x0, 4, &x)
                ioc> x
                value = 0xVALUE
RW is 0x0 for read and 0x1 for write
LVAR is the variable to read into or write from ADDRESS
sysBusToLocalAdrs(0xAMC, 0xaddress, &VARIABLE) Will tell you where a VME bus address is mapped to in VxWorks memory.
                ioc> x = 1
                ioc> sysBusToLocalAdrs(0x39, 0xBUS_ADDRESS, &x)
                ioc> x
                value = 0xLOCAL_ADDRESS
The Address Modifier Codes are found in vme.h
                0x09 A32 "user privilege"  Data
                0x0a A32 "user privilege"  program instruction fetch
                0x0b A32 "user privilege"  Block Transfer
               *0x0d A32 "supervisor priv" Data
                0x0e A32 "supervisor priv" program instruction fetch
                0x0f A32 "supervisor priv" Block transfer
                0x29 A16 "user privilege"  I/O
               *0x2d A16 "supervisor priv" I/O
                0x39 A24 "user privilege"  Data
                0x3a A24 "user privilege"  program instruction fetch
                0x3b A24 "user privilege"  Block transfer
               *0x3d A24 "supervisor priv" Data
		0x3e A24 "supervisor priv" program instruction fetch
                0x3f A24 "supervisor priv" Block transfer
        * = AMCs of choice
Also useful: Chethan Parameswariah's VxWorks Command Cheat Sheet.

For example:
We have a Struck scaler at address $a0 00 00 00. The first four bytes of address space are its control register. Bit "0" lights the lamp if set. We can light the user lamp thusly:

First, because we must read and write four bytes at a time, let's find out what's there before we mess with it:

  ioc> x = 0 ; vxMemProbe(0xa0000000, 0x0, 4, &x) ; x
  x = 0x6d5e28: value = 0 = 0x0
  value = 0 = 0x0
  x = 0x6d5e28: value = 9380232 = 0x8f2188
So we know that the four bytes are 00 8f 21 88. To light the user lamp, we set bit zero:
y = 0x008f2189 ; vxMemProbe(0xa0000000, 0x1, 4, &y)
Turning off the user lamp requires clearing the low bit:
y = 0x008f2188 ; vxMemProbe(0xa0000000, 0x1, 4, &y)
And we can read the module identification at an offset of 0x4:
x = 0 ; vxMemProbe(0xa0000004, 0x0, 4, &x) ; x
N.B. This is a case where "d 0xa0000000" will fail.
(Installing the EPICS Client) <-BACK | NEXT-> (Installing the EPICS Server)