Feeds:
Posts
Comments

Building SqueakSSL compiles just fine, but fails at the libtool stage when compiled as an internal plugin.

My hunch is that when compiled as an external plugin, the linking is deferred until the module is loaded by the VM. If that is the case, then it explains the primitive fail.

My approach was to compile it as an internal plugin as (by just giving it a try) I found what I thought was an #include error. It took me a while (too long really–I have a bad habit of not closely reading things) find the problem, but it happens after a successful compilation when a thing called “libtool” is run.

Here is the output of the error at that stage:

/bin/sh
[......]/debugCogSqueakSSL/unixbuild/bld/libtool --mode=link gcc -m32 -g -Og -msse2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DNDEBUG -DITIMER_HEARTBEAT=1 -DCOGMTVM=0 -DDEBUGVM=0 -DLSB_FIRST=1 -Wl,-z,now -export-dynamic -o squeak vm/vm.a AsynchFilePlugin/AsynchFilePlugin.a B2DPlugin/B2DPlugin.a BitBltPlugin/BitBltPlugin.a FilePlugin/FilePlugin.a SocketPlugin/SocketPlugin.a SqueakSSL/SqueakSSL.a MiscPrimitivePlugin/MiscPrimitivePlugin.a disabledPlugins.o version.o -lutil -ldl -lpthread -lm -lnsl -lpthread vm/vm.a

gcc -m32 -g -Og -msse2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -DNDEBUG -DITIMER_HEARTBEAT=1 -DCOGMTVM=0 -DDEBUGVM=0 -DLSB_FIRST=1 -Wl,-z -Wl,now -o squeak disabledPlugins.o version.o -Wl,--export-dynamic vm/vm.a AsynchFilePlugin/AsynchFilePlugin.a B2DPlugin/B2DPlugin.a BitBltPlugin/BitBltPlugin.a FilePlugin/FilePlugin.a SocketPlugin/SocketPlugin.a SqueakSSL/SqueakSSL.a MiscPrimitivePlugin/MiscPrimitivePlugin.a -lutil -ldl -lpthread -lm -lnsl -lpthread vm/vm.a

SqueakSSL/SqueakSSL.a(sqUnixOpenSSL.o): In function `sqCopyBioSSL':
[......]/debugCogSqueakSSL/platforms/unix/plugins/SqueakSSL/sqUnixOpenSSL.c:37: undefined reference to `BIO_ctrl_pending'

(more errors omitted for brevity. all of them are unresolved references to stuff in /usr/include/openssl/ssl.h . these errors do not happen at compile time, only linking stage )

Since discovering this, I have tried two things

1. compile on my pure 64 bit partition…this failed –we don’t have 64 bit Cog yet (: but hey…a guy can hope!

2. I installed openssl compat32 libs alongside the 64 bit versions…as shown below.
2.a I made some changes that I kinda-sorta-hoped would work in some strings in the script that had the substring “/lib64 ” but that did not work.

bash-4.2$ ls lib/*ssl*
lib/libssl.so.0 lib/libssl.so.0.9.8 lib/libssl.so.1 lib/libssl.so.1.0.0
bash-4.2$ ls lib64/*ssl*
lib64/libssl.so.0 lib64/libssl.so.0.9.8 lib64/libssl.so.1 lib64/libssl.so.1.0.0

I posted the above to the vm-dev mailing list and will see if anybody picks up on it. I will hit this again tomorrow!

Well, look what we have here from platforms/unix/vm/sqPlatformSpecific.h for the CogVM source tree.


#if defined(__GNUC__)
# if !defined(VM_LABEL)
# define VM_LABEL(foo) asm("\n.globl L" #foo "\nL" #foo ":")
# endif
#else
# if HAVE_ALLOCA_H
# include
# else
# ifdef _AIX
# pragma alloca
# else
# ifndef alloca /* predefined by HP cc +Olibcalls */
char *alloca();
# endif
# endif
# endif
#endif

indented #defines and #includes…I will have to research this a bit more, but if it is true, then putting these puppies and debugging them just got a whole lot easier.

I am stuck in the old code/build/debug loop of C while I try to learn how to debug the SqueakSSL plugin on Squeak.

The Plugin builds (but fails) when compiled as an external plugin in plugins.ext, but does not compile when moved to plugins.int as an internal plugin.

The reason it fails is that the include for #include “openssl/ssl.h” fails and many functions in the plugin are not referenceable (howdya like those c terms!)

I tried changing the #include statment through various contortions and those failed, so it looks like the quick fix method is not to be.

To simplify debugging, I like to remove clutter. Ergo, I created a debugCogClean directory and added to it the minimal directories and files needed for a functioning image (with monticello) to run.

The plugins needed are:


INTERNAL_PLUGINS = \
AsynchFilePlugin \
B2DPlugin \
BitBltPlugin \
FilePlugin \
SocketPlugin \
MiscPrimitivePlugin

the directory tree is:

|-- platforms
| |-- Cross
| | |-- BitBltPlugin
| | |-- plugins
| | | |-- AsynchFilePlugin
| | | |-- B3DAcceleratorPlugin
| | | |-- FilePlugin
| | | `-- SocketPlugin
| | `-- vm
| `-- unix
| |-- config
| |-- doc
| | `-- HowToBuildFromSource.html
| |-- plugins
| | |-- AsynchFilePlugin
| | |-- B3DAcceleratorPlugin
| | | `-- zzz
| | |-- FilePlugin
| | `-- SocketPlugin
| |-- vm
| |-- vm-display-Quartz
| | |-- Resources
| | | `-- SqueakHelp
| | | `-- images
| | `-- zzz
| |-- vm-display-X11
| |-- vm-display-custom
| |-- vm-display-fbdev
| |-- vm-display-null
| |-- vm-sound-ALSA
| |-- vm-sound-MacOSX
| |-- vm-sound-NAS
| |-- vm-sound-OSS
| |-- vm-sound-Sun
| |-- vm-sound-custom
| `-- vm-sound-null
|-- src
| |-- plugins
| | |-- AsynchFilePlugin
| | |-- B2DPlugin
| | |-- B3DAcceleratorPlugin
| | |-- BitBltPlugin
| | |-- FilePlugin
| | |-- MiscPrimitivePlugin
| | `-- SocketPlugin
| `-- vm
`-- unixbuild
`-- bld
|-- AsynchFilePlugin
|-- B2DPlugin
|-- BitBltPlugin
|-- FilePlugin
|-- MiscPrimitivePlugin
|-- SocketPlugin
|-- vm
|-- vm-display-X11
|-- vm-display-null
|-- vm-sound-ALSA
|-- vm-sound-OSS
`-- vm-sound-null

This will be my base point for debugging plugins. Now, for the SqueakSSL plugin, I will create a directory debugCogSqueakSSL and copy in that clean tree. then jus t add the relevant SqueakSSL files.

Hopefully I can get this done soon.

I just completed this tutorial:

http://stephane.ducasse.free.fr/FreeBooks/CollectiveNBlueBook/greenberg.pdf

I have written my first plugin! Woot!

Okey-dokey…now on to fixing the SqueakSSL plugin.

There are 3 exercises I went through.

Build 64 bit native VM that runs a 32 bit image
Build a 64 bit native VM compiled to support a 64 bit image downloaded from the Jenkins build server
Generate my own 64 bit image and run that on the VM built in 2 above.

This post is the quick-n-dirty of exercise 3 of the 3.

Please read the previous 2 posts in this series at:

http://timmydosmalltalk.wordpress.com/2014/03/13/howto-build-a-64-native-standardvm-running-32-bit-image-on-slackware-linux-14-1-with-32-bit-compat-libs/

and:

http://timmydosmalltalk.wordpress.com/2014/03/13/howto-build-a-64-native-standard-vm-running-64-bit-image-from-jenkins-build-server/

This exercise assumes you have completed the “64 native Standard VM running 64 bit image from Jenkins build server” exercise
and have an Interpreter VM capable of running a 64 bit image. If you don’t then this will not work.

What we are going to do here is use the SystemTracer on the All-In-One app to generate the 64 bit .image and .changes file
instead of downloading it from Jenkins. I am using my all-in-one install from the above “64 native StandardVM running 32 bit image on Slackware Linux 14.1 with 32 bit compat libs” exercise

#LAUNCH ALL-IN-ONE, INSTALL SystemTracer,
cd ~/usr/src/smalltalk/buildStandardVM32/Squeak-4.5-All-in-One.app
./squeak.sh

Toolbar menu->Tools->Monticello Browser
+Repository->HTTP
MCHttpRepository
location: ‘http://www.squeaksource.com/SystemTracing’
user: ‘squeak’
password: ‘squeak’

Open->SystemTracing-dtl.25.mcz (or latest)->Load

Open a Browser and look at class comment for SystemTracer64 (cut-n-paste below)
traceTo64Script
“This is a script that can be filed in to a 32-bit image to trace that image to 64-bit
format. The SystemTracing package is loaded from a local file, and is removed
from the image when tracing is complete. Save the script to a file, then start the
image to be traced, providing the script as a parameter on the command line.

Create the script file by evaluating this:

| fs |
fs := FileStream newFileNamed: ‘TraceTo64.st’.
fs nextPutAll: (SystemTracer64 traceTo64Script).
fs close.

Get a copy of the a Monticello source file for SystemTracer from the repository
at http://www.squeaksource.com/SystemTracing and place it in your working directory.

Then on the command line:
$ squeak myImageToBeTraced TraceTo64.st”

#GENERATE IMAGE AND CHANGES FILE WITH THE SCRIPT, DO INITIAL LAUNCH IN ORIGINAL DIRECTORY
cd Contents/Resources/
pwd /home/tty/usr/src/smalltalk/buildStandardVM32/Squeak-4.5-All-in-One.app/Contents/Resources
cp package-cache/SystemTracing-dtl.25.mcz ./
ls ls *.st
TraceTo64.st
ls *.mcz
SystemTracing-dtl.25.mcz
../Linux-i686/bin/squeak Squeak4.5-13680.image TraceTo64.st (This will launch squeak and generate the .image and .changes)
ls *.image
Squeak4.5-13680-64.image Squeak4.5-13680.image
ls Squeak4.5-13680-64*
Squeak4.5-13680-64.changes Squeak4.5-13680-64.image (ok, we have the new image and changes. It is important to first launch them in the current directory or else it gets confused)

export LD_LIBRARY_PATH=~/usr/local/standardVM64/lib
/home/tty/usr/local/standardVM64/lib/squeak/4.12.13-_64bit/squeakvm64 -vm-sound-ALSA -vm-display-X11 -xshm Squeak4.5-13680-64.image &

(at this point I get a blank desktop)

#CONFIGURE RUN ENVIRONMENT AND LAUNCH

cd /home/tty/usr/src/smalltalk/
mkdir runStandardVM64Traced
cd runStandardVM64Traced
(NOTE: we need .image, .changes and .sources files. we use the .sources …)
cp ../runStandardVM32/SqueakV41.sources ./
mv ../buildStandardVM32/Squeak-4.5-All-in-One.app/Contents/Resources/Squeak*64.* ./
ls
SqueakV41.sources Squeak4.5-13680-64.changes Squeak4.5-13680-64.image

#make a bash script startStandardVM64Traced.sh with the following contents (we are re-using the vm we created in the previous exercise)
#!/bin/bash
export LD_LIBRARY_PATH=~/usr/local/standardVM64/lib
/home/tty/usr/local/standardVM64/lib/squeak/4.12.13-_64bit/squeakvm64 -vm-sound-ALSA -vm-display-X11 -xshm Squeak4.5-13680-64.image

chmod 0755 startStandardVM64Traced.sh
./startStandardVM64Traced.sh

Toolbar menu->Help->About This System. (not working)

There are 3 exercises I went through.

  1. Build 64 bit native VM that runs a 32 bit image
  2. Build a 64 bit native VM compiled to support a 64 bit image downloaded from the Jenkins build server
  3. Generate my own 64 bit image and run that on the VM built in 2 above.

This post is the quick-n-dirty of exercise 2 of the 3.

Please read the previous post in this series at: http://timmydosmalltalk.wordpress.com/2014/03/13/howto-build-a-64-native-standardvm-running-32-bit-image-on-slackware-linux-14-1-with-32-bit-compat-libs/

In this exercise we are going to compile an Standard Interpreter VM capable of powering a 64 bit image. We will download a pre-traced 64 bit image from the Jenkins server. In the next exercise, we will generate our own 64 bit image.

Steps marked (no change) are explained in the previous exercise linked above.

#OVERVIEW (no change)

#PREREQUISITS AND ASSUMPTIONS (no change)

#CREATE WORKING DIRECTORY
mkdir ~/usr/src/smalltalk/buildStandardVM64

#DOWNLOAD SQUEAK TO BUILD DIRECTORY
cd ~/usr/src/smalltalk/buildStandardVM64
Download http://ftp.squeak.org/4.5/Squeak-4.5-All-in-One.zip to ~/usr/src/smalltalk/buildStandardVM64
unzip Squeak-4.5-All-in-One.zip

#DOWNLOAD BUILD ENVIRONMENT TO SAME DIRECTORY AS .image
cd ~/usr/src/smalltalk/buildStandardVM64/Squeak-4.5-All-in-One.app/Contents/Resources
svn co http://squeakvm.org/svn/squeak/trunk
ln -s trunk/platforms platforms
mkdir build
mkdir src

#LAUNCH ALL-IN-ONE
cd ~/usr/src/smalltalk/buildStandardVM64/Squeak-4.5-All-in-One.app
./squeak.sh

#INSTALL VMMAKER (no change)

#CONFIGURE AND RUN VMMAKER TO GENERATE C SOURCE CODE: TWO METHODS (no change)

#CREATE INSTALL DIRECTORY, CONFIGURE,MAKE,MAKE-INSTALL
cd ~/usr/src/smalltalk/buildStandardVM64/Squeak-4.5-All-in-One.app/Contents/Resources/build
../platforms/unix/cmake/configure –help
mkdir /home/tty/usr/local/standardVM64
../platforms/unix/cmake/configure –prefix=/home/tty/usr/local/standardVM64 –without-gl –image64 <–NOTE THE –image64 flag and the install directory
make
make install

file /home/tty/usr/local/standardVM64/lib/squeak/4.12.13-_64bit/squeakvm64
/home/tty/usr/local/standardVM64/lib/squeak/4.12.13-_64bit/squeakvm64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
(file command shows it is a native 64 app)

#OPTIONAL reboot to pure 64 bit system if you have one.

#CONFIGURE RUN ENVIRONMENT AND LAUNCH

cd /home/tty/usr/src/smalltalk/
mkdir runStandardVM64
cd runStandardVM64
(NOTE: we need .image, .changes and .sources files. we use the .sources from the 32bit build exercise and download the image and changes from jenkins server)
cp ../runStandardVM32/SqueakV41.sources ./
download .image and .changes from

http://build.squeak.org/job/Squeak2064-bit20image/lastSuccessfulBuild/artifact/TrunkImage-64.changes

http://build.squeak.org/job/Squeak2064-bit20image/lastSuccessfulBuild/artifact/TrunkImage-64.image

ls
SqueakV41.sources TrunkImage-64.changes TrunkImage-64.image (N.B. we now have a pristine setup. executables and libs are in ~/usr/local/standardVM64 annd bare minimum here image,changes, sources)

#make a bash script startStandardVM64.sh with the following contents
#!/bin/bash
export LD_LIBRARY_PATH=~/usr/local/standardVM64/lib
/home/tty/usr/local/standardVM64/lib/squeak/4.12.13-_64bit/squeakvm64 -vm-sound-ALSA -vm-display-X11 -xshm TrunkImage-64.image

chmod 0755 startStandardVM64.sh
./startStandardVM64.sh

Toolbar menu->Help->About This System.

Image
—–
/home/tty/usr/src/smalltalk/runStandardVM64/TrunkImage-64.image
Squeak4.5
latest update: #13685
Current Change Set: Unnamed1
Image format 68002 (64 bit)

Virtual Machine
—————
/home/tty/usr/local/standardVM64/lib/squeak/4.12.13-_64bit/squeakvm64
Squeak4.5 of 19 February 2014 [latest update: #13680]
Unix built on Mar 13 2014 12:26:32 Compiler: 4.8.2
platform sources revision
VMMaker versionString 4.12.13

There are currently three version of VMMaker at source.squeak.org

  1. VMMaker (legacy package for building the old Standard Interpreter)
  2. VMMaker.oscog (Eliot Miranda’s current work supporting Stackinterpreter, Cog, newspeak, Spur etc)
  3. VMMaker-oscog (Pharo branch of VMMaker.oscog)

By luck or design, VMMaker uses CMAKE for its config/build work and somehow CMAKE knows enough to detect my system is linux64 and ignores
the compat32 libs. This gives me a native 64 bit virtual machine. Since I want to port StackInterpreter and eventually Cog to native 64, I thought
it would be important to get the feel of the old VMMaker process before advancing to the StackInterpreter in VMMaker.oscog

There are 3 exercises I went through.

  1. Build 64 bit native VM that runs a 32 bit image
  2. Build a 64 bit native VM compiled to support a 64 bit image downloaded from the Jenkins build server
  3. Generate my own 64 bit image and run that on the VM built in 2 above.

This post is the quick-n-dirty of exercise 1 of the 3.

#OVERVIEW
The following is an exercise to gain experience in generating a vm and a 64 bit image.
We will be using a current system with a Cog VM to generate an out-dated, slower Interpreter VM.

See David T. Lewis’ work: How to build a standard VM with Squeak on Linux at href=”http://wiki.squeak.org/squeak/6177″>http://wiki.squeak.org/squeak/6177
Below are my short-hand notes on his work.

#PREREQUISITS AND ASSUMPTIONS

32-bit compatability libs installed and working (Slackware used Alien-Bob compat32, Ubuntu etc have their own flavors of this)
cmake installed.
svn installed
in instruction below….
replace Squeak4.5-* with your actual version
replace VMMaker-dtl.345 with the latest version of VMMaker from Monticello
replace update-dtl.15.mcm with the latest version of VMMaker from Monticello
replace /home/tty/ with your home directory
replace ~/usr/src/smalltalk/buildStandardVM32 with your preference
optional: I have a dual boot system. One boot is Slackware 14.1 with the 32 bit compat libs, the other is a pure 64 bit system with no compat libs.
both boots share the same /home directory which is mounted on its own disk.

#CREATE WORKING DIRECTORY
mkdir ~/usr/src/smalltalk/buildStandardVM32
cd ~/usr/src/smalltalk/buildStandardVM32

#DOWNLOAD SQUEAK TO BUILD DIRECTORY
Download http://ftp.squeak.org/4.5/Squeak-4.5-All-in-One.zip to ~/usr/src/smalltalk/buildStandardVM32
unzip Squeak-4.5-All-in-One.zip

#DOWNLOAD BUILD ENVIRONMENT TO SAME DIRECTORY AS .image
cd ~/usr/src/smalltalk/buildStandardVM32/Squeak-4.5-All-in-One.app/Contents/Resources
svn co http://squeakvm.org/svn/squeak/trunk
ln -s trunk/platforms platforms
mkdir build
mkdir src

#Launch all-in-one
cd ~/usr/src/smalltalk/buildStandardVM32/Squeak-4.5-All-in-One.app
./squeak.sh

#Install VMMaker
Tools-> Monticello Browser
+Repository->HTTP
MCHttpRepository
location: ‘http://source.squeak.org/VMMaker&#8217;
user: ”
password: ”
Open->(scroll down in left hand pane to..)update->update-dtl.15.mcm->Load (this installs lates VMMaker and dependencies)
(N.B. VMMaker is standard interpreter, VMMaker-oscog is pharo branch of cog, VMMaker.oscog is squeak branch of Cog, we are building the standard interpreter, so use VMMaker)
Save your image.

#CONFIGURE AND RUN VMMAKER TO GENERATE C SOURCE CODE: TWO METHODS

#RUNNING VMMaker GUI method
Toolbar menu->Apps->VMMaker

#enter fields in gui:
Interpreter class name: Interpreter
Path to platforms code: /home/tty/usr/src/smalltalk/buildStandardVM32/Squeak-4.5-All-in-One.app/Contents/Resourcesplatforms <–N.B. this is the soft link you created earlier
path to generated sources:/home/tty/usr/src/smalltalk/buildStandardVM32/Squeak-4.5-All-in-One.app/Contents/Resources/src

#drag-n-drop plugins (these worked for me, others may as well, your mileage may vary. I am just keeping it simple for pedantic purposes)
Internal plugins
BalloonEnginePlugin
BitBltSimulation
DeflatePlugin
FilePlugin
LargeIntegersPlugin
LocalePlugin
SecurityPlugin
SocketPlugin

External Plugins
UUIDPlugin
UnixAioPlugin
UnixOSProcessPlugin
XDisplayControlPlugin

#Buttons to push
[Clean out] (next to Path to generated sources:) cleans out the directory
Generate: [Entire] (Second row from top, wait for the whirring,clicking and blinkenlights to stop)
[Save] (saves this configuration to a file for loading later)

#RUNNING VMMaker scripting method
Open Workspace and DoIt

(VMMaker default
initializeInternal: #(BalloonEnginePlugin BitBltSimulation DeflatePlugin FilePlugin LargeIntegersPlugin LocalePlugin SecurityPlugin SocketPlugin)
external: #(SocketPlugin ZipPlugin B2DPlugin)) generateEntire

save and quit Squeak.

#CREATE INSTALL DIRECTORY, CONFIGURE,MAKE,MAKE-INSTALL
cd ~/usr/src/smalltalk/buildStandardVM32/Squeak-4.5-All-in-One.app/Contents/Resources/build
../platforms/unix/cmake/configure –help
mkdir /home/tty/usr/local/standardVM32 <–where I want the new vm executable to be installed, your path is your preference used in –prefix= below
../platforms/unix/cmake/configure –prefix=/home/tty/usr/local/standardVM32 –without-gl <
make
make install

file /home/tty/usr/local/standardVM32/lib/squeak/4.12.13-/squeakvm
/home/tty/usr/local/standardVM32/lib/squeak/4.12.13-/squeakvm: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), not stripped
(file command shows it is a native 64 app)

#OPTIONAL reboot to pure 64 bit system if you have one.

#CONFIGURE RUN ENVIRONMENT AND LAUNCH (N.B. I like to keep my build and run environments separate. )
cd /home/tty/usr/src/smalltalk/
mkdir runStandardVM32
cd runStandardVM32
cp ../buildStandardVM32/Squeak-4.5-All-in-One.zip ./
unzip Squeak-4.5-All-in-One.zip
mv Squeak-4.5-All-in-One.app/Contents/Resources/Squeak4.5-13680.* ./
mv Squeak-4.5-All-in-One.app/Contents/Resources/SqueakV41.sources ./
rm -Rf Squeak-4.5-All-in-One.app/
rm -Rf __MACOSX
mv Squeak-4.5-All-in-One.zip ~/
ls
Squeak4.5-13680.changes Squeak4.5-13680.image SqueakV41.sources (N.B. we now have a pristine setup. executables and libs are in ~/usr/local/standardVM32 and bare minimum here image,changes, sources)

#make a bash script startStandardVM32.sh with the following contents
#!/bin/bash
export LD_LIBRARY_PATH=~/usr/local/standardVM32/lib
/home/tty/usr/local/standardVM32/lib/squeak/4.12.13-/squeakvm -vm-sound-ALSA -vm-display-X11 -xshm Squeak4.5-13680.image

chmod 0755 startStandardVM32.sh
./startStandardVM32.sh

Toolbar menu->Help->About This System.

Follow

Get every new post delivered to your Inbox.