Perl: Initializing 2-dimensional arrays using the repeat operator 'x'

I was trying to initialize a two dimensional Perl array with the repeat operator 'x', i.e.:

my @array = ([]) x 2;

Then I did something to the effect of,

for (my $i=0; $i<6; ++$i) { push(@{$array[$i % 2]}, 1); }

Thinking that I will create the equivalent of the following structure:

@array = ( [ 1, 1, 1 ], [ 1, 1, 1 ] );

However, doing:

print "@{$array[0]}\n";
print "@{$array[1]}\n";

gave,

1 1 1 1 1 1
1 1 1 1 1 1


How did that happen?


It seems that the left argument to the repeat operator, in this case, ( [] ) is evaluated before the repetition. Hence the initialization of @array achieved an effect similar to the following:


my $aRef = [];
my @array = ( $aRef, $aRef );


That is, the same reference to an empty array is repeated instead of a fresh reference. Hence the same array was pushed 6 times. 


As it turns out, it is better not to initialize the array at all and let the dynamic type system figure out the element type from a push,

my @array = ();
for (my $i=0; $i<6; ++$i) { push(@{$array[$i % 2]}, 1); }


Which resulted in the correct outcome, ( [ 1, 1, 1 ], [ 1, 1, 1 ] ), i.e., an array with two distinct array references.

Bash scripting: parallel process control with 'bash' and 'xargs'

I have been attempting to run bash commands in parallel on Windows with bash given by msys with some form of control over the number of processes spawned. With this setup I do not have access to the parallel command.

For example, we can always specify the number of processes for compilation to the make command using:

$ make -j4

that uses 4 parallel processes and no more. After much trial and error, I finally figured out how multiple arbitrary commands can be run in the same way with a similar kind of control.

Let us presume we have a command file with one line per command. For example, I am trying to build different machine learning models to predict outcomes on various datasets in parallel on a multi-core machine using WEKA. Hence I have a text file, cmd.txt, prepared by a script that contains lines like:

$JBIN -Xmx3g weka.classifiers.trees.J48 -C 0.25 -M 2 -A -i -t result-1-1/filter6-weka-train.arff -T result-1-1/filter6-weka-test.arff -p 0 -d result-1-1/filter6-J48.model > result-1-1/filter6-J48-report.txt
$JBIN -Xmx3g weka.classifiers.functions.LibSVM -S 0 -K 2 -D 3 -G 0.0 -R 0.0 -N 0.5 -M 1000.0 -C 1000000.0 -E 0.0010 -P 0.1 -Z -o -i -t result-1-1/filter6-weka-train.arff -T result-1-1/filter6-weka-test.arff -p 0 -d result-1-1/filter6-SVM.model > result-1-1/filter6-SVM-report.txt
$JBIN -Xmx3g weka.classifiers.functions.LibSVM -S 0 -K 2 -D 3 -G 0.0 -R 0.0 -N 0.5 -M 1000.0 -C 1000000.0 -E 0.0010 -P 0.1 -Z -W '1 2' -o -i -t result-1-1/filter6-weka-train.arff -T result-1-1/filter6-weka-test.arff -p 0 -d result-1-1/filter6-SVM-w-1-2.model > result-1-1/filter6-SVM-w-1-2-report.txt
$JBIN -Xmx3g weka.classifiers.functions.LibSVM -S 0 -K 2 -D 3 -G 0.0 -R 0.0 -N 0.5 -M 1000.0 -C 1000000.0 -E 0.0010 -P 0.1 -Z -W '2 1' -o -i -t result-1-1/filter6-weka-train.arff -T result-1-1/filter6-weka-test.arff -p 0 -d result-1-1/filter6-SVM-w-2-1.model > result-1-1/filter6-SVM-w-2-1-report.txt
$JBIN -Xmx3g weka.classifiers.trees.J48 -C 0.25 -M 2 -A -i -t result-1-1/filter9-weka-train.arff -T result-1-1/filter9-weka-test.arff -p 0 -d result-1-1/filter9-J48.model > result-1-1/filter9-J48-report.txt
$JBIN -Xmx3g weka.classifiers.functions.LibSVM -S 0 -K 2 -D 3 -G 0.0 -R 0.0 -N 0.5 -M 1000.0 -C 1000000.0 -E 0.0010 -P 0.1 -Z -o -i -t result-1-1/filter9-weka-train.arff -T result-1-1/filter9-weka-test.arff -p 0 -d result-1-1/filter9-SVM.model > result-1-1/filter9-SVM-report.txt
$JBIN -Xmx3g weka.classifiers.functions.LibSVM -S 0 -K 2 -D 3 -G 0.0 -R 0.0 -N 0.5 -M 1000.0 -C 1000000.0 -E 0.0010 -P 0.1 -Z -W '1 2' -o -i -t result-1-1/filter9-weka-train.arff -T result-1-1/filter9-weka-test.arff -p 0 -d result-1-1/filter9-SVM-w-1-2.model > result-1-1/filter9-SVM-w-1-2-report.txt
$JBIN -Xmx3g weka.classifiers.functions.LibSVM -S 0 -K 2 -D 3 -G 0.0 -R 0.0 -N 0.5 -M 1000.0 -C 1000000.0 -E 0.0010 -P 0.1 -Z -W '2 1' -o -i -t result-1-1/filter9-weka-train.arff -T result-1-1/filter9-weka-test.arff -p 0 -d result-1-1/filter9-SVM-w-2-1.model > result-1-1/filter9-SVM-w-2-1-report.txt

where $JBIN is an environment variable that points to the java bin. Now to run these in parallel but with a limit on the number of processes, use the xargs command to split the input lines as follows:

$ cat cmd.txt | xargs -0 -d '\n' -L 1 -I {} -P 3 bash -c "eval \"{}\""

The options used are:

1. -0 to retain quotes in the input line and presume arguments are terminated as \0 characters
2. -d '\n' to set newline as the delimiter between arguments, overriding \0 in the previous point
3. -L 1 to read one line at a time
4. -I {} to set parenthesis as a replacement string to substitute the argument read, in this case an entire line
5. -P 3 to limit to a maximum of 3 processes
6. bash -c "eval \"{}\"" to execute the substituted command within bash

And that is it. It works as long as the commands are on a single line. I have yet to test it on commands spanning multiple lines.

Changing SSH username for svn+ssh SVN access method

Having setup SVN over SSH for work purposes one frustration I faced was the issue of switching users for a repository to test path based access control. First I checked out the working copy using a read-only user via:

$>svn co svn+ssh://readonlyuser@myserver/repository/myrepo

Tried to commit a change and it was not allowed as expected. Now to test the read/write user without deleting and rechecking out with the new user. The "readonlyuser" has been cached in the checked out repository. How do I change to a different user?

It took me a long while to figure this out as the following methods do not work. Do the initial checkout by specifying the --username option and hope that we can commit with a different username later:

$>svn co --username readonlyuser svn+ssh://myserver/repository/myrepo

$>svn ci --username readwriteuser

This fails as the --username option is ignored when using the svn+ssh method and the current logged in user account on the client machine will be used as the svn user instead.

Next try using the svn switch command after checking out:

$>svn co svn+ssh://readonlyuser@myserver/repository/myrepo

$>svn switch svn+ssh://readwriteuser@myserver/repository/myrepo

This fails miserably as well with the useless error message:

svn: 'svn+ssh://readonlyuser@myserver/repository/myrepo'
is not the same repository as
'svn+ssh://readwriteuser@myserver/repository/myrepo'

Finally the way that actually works is to use the --relocate option with switch:

$>svn switch --relocate svn+ssh://readonlyuser@myserver/repository/myrepo svn+ssh://readwriteuser@myserver/repository/myrepo

This prompts for the password of the write user and switches the user successfully.

Overlay Images

I have decided to attempt to use CSS to overlay images by following this as a guide. Looks like it works. =D

Now I can display panoramas that are longer than the blog width.

Mouse over icon to see

Hidden multiple argument BAT file execution with VBS

I have been writing a program that calls external programs using batch files in windows. Every time the call is made, the ugly command line box appears. After searching the net it seems that a simple way would be to use Visual Basic Script (VBS) to call the batch file in a special shell. Since the VBS interpreter is available on WinXP and above systems, I decided to try this vbs script that I found somewhere off the net.

Set WshShell = CreateObject("WScript.Shell")
WshShell.Run chr(34) & WScript.Arguments(0) & Chr(34), 0
Set WshShell = Nothing


Put the above code in a file called "invi.vbs" and then call the following from command line:

$>wscript.exe "invi.vbs" "mybatchfile.bat"


The batch file runs without a window! Thinking that this was great, I tried to run a batch file with arguments. Now that fails somehow because the interpreter seems to think that the entire WScript.Arguments(0) is a program name, i.e. "mybatchfile.bat arg1" will be treated as a program name instead of a batch file with a single argument.

Having not written VB since eons ago, I used my limited knowledge to concatenate the WScript.Arguments into a string. Somehow it just worked after replacing "invi.vbs" with my edits below.

Set WshShell = CreateObject("WScript.Shell")
dim toexec
For Each x in WScript.Arguments
   toexec = toexec & x & " "
Next
WshShell.Run toexec, 0, 1
Set WshShell = Nothing


Then, running it without quotes for the batch file, i.e.:

$>wscript.exe "invi.vbs" mybatchfile.bat arg1 arg2

MinGW to MSVC -- the case of rint()

I have recently been working on shifting a project over from compiling with the MinGW implementation of the GNU C++ compiler to the Microsoft Visual C++ (MSVC) compiler. The most glaring difference is the lack of the rint() function in MSVC. rint(x) takes a double, x, and returns the double value of the closest integer to x. Being a lazy bum I opened the math.h file for MinGW to copy the rint() function to a global win32 compatibility header. Unfortunately, it is written using an assembly instruction and a bunch of macros. Not wanting to copy the macros over, I did the next lazy thing and used Google.

More unfortunately for me, I copied a version of rint() that was not compatible with the implementation in MinGW. I had:

double rint(double x)
{
     return floor(x < 0.0 ? x - 0.5 : x + 0.5);
}

It turns out, that this is wrong as -1.5 should be rounded to -1.0 and not -2.0. After wasting a few hours debugging, I finally realised it should be the much simpler:

double rint(double x)
{
     return floor(x + 0.5);
}

So much for being a lazy bum...

JVM Max Heap Size on WinXP 32-bit

I have been trying to run some memory intensive java program on a WinXP Pro system with 2gb of RAM. Unfortunately, I could never reserve more than 1.4gb of memory on it for the JVM heap (using -Xmx). Frustrated, I turned to a 32-bit Linux server on Fedora Core 5. To my surprise, I managed to reserve 1.8gb. It appears that on 32-bit systems the maximum one can reserve is slightly less than 2gb and because of OS constraints, different OSes (hence different JVMs) have different limits on max heap size. WinXP Pro only allows from 1.4gb to 1.6gb. Anything more than 2gb requires a 64-bit machine as stated in Sun's FAQ here.

C++ scanf and fscanf woes

I have been doing a whooping lot of C++ coding recently and one of the things that irked me the most was the scanf/fscanf function. For starters, one can write out double type variables to the fprintf and printf functions. Assuming that, since this can be done, I attempted to use fscanf to read double types from a file. As it turns out the double types cannot be read -- no value is stored in the double variable.

Why? Well apparently the scan functions can only read float types with the "%f" mask. So a float variable is required to read the float and then a static cast must be performed to turn it into a double. To read a double the mask should be "%lf". The most hard to detect part is the compiler accepts a double type as a legitimate type for the scanf/fscanf functions, so some form of warnings must be switched on to detect that. To further complicate things, printf functions use "%f" to print doubles.

Caught in a rat trap

Now, lately I have been working on a project that involves image processing. For the last couple of hours my program has been causing my PC speaker to beep when I tried to dump an image to a file "con.bmp". Furthermore symbols started to appear on my command line for no reason it seems. Extremely frustrated at the lack of progress I finally decided to post it as a question on a developer forum.

The reply I got from a kind expert was kind shocking. There was nothing wrong with my program. I was totally smacked by M$'s WinXP. Turns out that WinXP does not allow files to be called "con" with any extension for that matter (e.g. "con.bmp"). These files are reserved device files. Unlike Linux where the device files are kept in a special place, in WinXP, they can be accessed everywhere (that is why they have to be reserved)! What is worse is that "con" and its extensions are not the only reserved file names.

"Do not use the following reserved device names for the name of a file: CON, PRN, AUX, NUL, COM1, COM2, COM3, COM4, COM5, COM6, COM7, COM8, COM9, LPT1, LPT2, LPT3, LPT4, LPT5, LPT6, LPT7, LPT8, and LPT9. Also avoid these names followed by an extension, for example, NUL.tx7.

Windows NT: CLOCK$ is also a reserved device name." -- msdn

Financial Noob's Discoveries Part 2

Problem: Given a loan with an interest rate compounded daily, a target duration to completely repay the loan, and the fixed interval to make payments, how much should one pay per interval to completely repay the loan by the target duration?

Solution:

Financial Noob's Discoveries

Recently I have been thinking about the following problem: Given a loan with an annual interest rate that is compounded daily, and given a payment to be made at fixed intervals, how much money is actually paid to recover the loan and over how long? Hence I came up with the simple calculator below (thanks to jw for pointing out the function to use; this should run slightly faster if the log operation in javascript is implemented in less than O(n), otherwise no speed gain):

Now, the harder question is finding the converse: given a loan with an interest rate compounded daily, a target duration to completely repay the loan, and the fixed interval to make payments, how much should one pay per interval to completely repay the loan by the target duration?

New Version of Slide Show Widget

Finally completed a new version of the Slide Show widget here. As previously mentioned, this widget is fully cut and paste with minimal changes necessary. There are two control parameters at the near the end of the code:

picObject.slideit(); }) (1000, 1); /*]]>*/

The green parameter indicates the duration in milliseconds that each picture will display before the next. A value of zero means the picture will only swap when the mouse is moved over it, i.e. like the banner above. The red parameter indicates if clicking on the picture will open a link to the picture. A value of 1 means yes (i.e. the slide show on the right) and a value of zero means no (i.e. the banner).

Somehow I cannot get the blogger API to share the widget properly using a form, it keeps scrambling up the code and displaying it as a Text/HTML/JavaScript widget when it should be a TextList widget =(.

JavaScript Total Re-discovery

After trying out JavaScript and reading this, I made a shocking discovery for myself. All along my impression of JavaScript was this language used to write some hasty gimmicks for web pages. After all it is just a scripting language, how advanced can it be?

I have been totally debunked. JavaScript is actually more like Scheme and Lisp than JAVA itself! And the even more amazing thing is that every modern web browser is an interpreter almost as powerful as the Scheme interpreter. JavaScript is a functional programming language extended with records and imperative constructs.

Now for the less technical folks, what does all these jibberish mean? It basically means that I can now create widgets that contain JavaScript without fear of clashing declarations with another widget (written by someone else) on the same HTML page. In other words keeping true to the idea of a widget being an independent GUI component that is not affected by and does not affect other widgets.

Finally becoming less noob. Look out for cut and paste widgets, almost no configurations necessary, from me soon.

New SlideShow Widget

I have finally created my first widget and I must say it was a horrendous experience. The code is available at my code testing blog here. The main trouble with the code was declaring objects in JavaScript. For some reason, over the past two years, numerous sites sprang up each claiming different techniques for declaring objects and their members/methods. After trying out numerous, I finally found one that works.

To understand what it does, move your mouse over my banner on top. That is the default behavior. Modifying this.slideshowspeed=0; to a value greater than zero will set the pictures to swap at different durations, i.e. 2000 for 2 seconds.

The widget is actually a hack of a TextList widget. This is to make use of the same interface provided by blogger to add URLs of pictures to display. This can be seen on the code testing blog mentioned earlier. One drawback of this hack is that for now it does not resize pictures to fit.

The reason behind creating a JavaScript Object, PicObject(), is to encapsulate the variables used in the script. This will allow multiple instances of this widget in a single blog. However two modifications have to be made to allow that:

1. The variable holding the object, "pickSwap" must be renamed throughout the code.
2. The image tag's name attribute, "slide", must be renamed throughout the code.

An example of two of these widgets operating is shown for the time being with some DotA icons on the right.