Sparklines: satisfaction and disappointment

Could go into a whole diatribe about the paradoxical human condition of conflicting feelings but will keep it simple. Have achieved what I set out to do: and -- sparkline the bank's account opening activity during a given period. Well, have achieved it roughly, anyway. But generating sparklines is a big hassle: code for Office 2000 or XP and above? (2000 doesn't have a feature which makes the user's life a hell of a lot easier.) When generating multiple sparklines at the same time, scale them all to the same scale or different (as they are above)? Currently I'm coding for MSO2000, though there's no guarantee the code will actually work; and scaling to different scales even when generating in the same batch. Will have to change both these settings because the alternatives are so much more helpful.

On the plus side, wrote a clever little toolbar that manages generated sparklines -- i.e. selecting and deleting them -- almost well enough to be called a sparkline manager.

Here are the macros that pull in and parse the plain-text reports generated daily by the bank's software:

Sub inputData(fname, aDoc)
  Set fso = CreateObject("Scripting.FileSystemObject")
  Set fin = fso.OpenTextFile(fname, 1)
  branchNamesStore = ""
  openingCountsStore = ""
  printingCounts = False
  weHaveTabs = False

  Do While fin.AtEndOfStream <> True
      aLine = fin.ReadLine
   
      Dim branchNamePoses(5)
      branchNamePoses(0) = InStr(1, aLine, " BRANCH       ")
      branchNamePoses(1) = InStr(1, aLine, " BRANCH" + vbTab)
      branchNamePoses(2) = InStr(1, aLine, " BRAN       ")
      branchNamePoses(3) = InStr(1, aLine, " BRAN" + vbTab)
      branchNamePoses(4) = InStr(1, aLine, " BRANC      ")
      branchNamePoses(5) = InStr(1, aLine, " BRANC" + vbTab)
   
      numAcctsPos = InStr(1, aLine, "OPENED :")
      lineHasComma = InStr(1, aLine, ",")
      branchName = ""
      branchNameTemp = ""
      numAcctsStr = ""

      For posCounter = 0 To UBound(branchNamePoses)
          If branchNamePoses(posCounter) <> 0 And lineHasComma = 0 Then
              If posCounter Mod 2 = 0 Then ' No tabs in this file
                  i = branchNamePoses(posCounter) - 1
                  Do While Mid(aLine, i, 1) <> " "
                      branchName = branchName & Mid(aLine, i, 1)
                      i = i - 1
                  Loop
                  branchNameTemp = branchName
                  branchName = StrReverse(branchNameTemp)
                  branchNamesStore = branchNamesStore & " " & branchName
              Else ' Tabs in the file
                  weHaveTabs = True
                  i = branchNamePoses(posCounter) - 1
                  Do While Mid(aLine, i, 1) <> vbTab
                      branchName = branchName & Mid(aLine, i, 1)
                      i = i - 1
                  Loop
                  branchNameTemp = branchName
                  branchName = StrReverse(branchNameTemp)
                  branchNamesStore = branchNamesStore & " " & branchName
              End If
          ElseIf InStr(1, aLine, "HEAD OFFICE") And lineHasComma = 0 And Not InStr(1, branchNamesStore, "HEADOFFICE") Then
              branchNamesStore = branchNamesStore & " HEADOFFICE"
              Exit For
          End If
      Next posCounter
   
      If numAcctsPos <> 0 Then
          i = numAcctsPos + 9
          Do While i < Len(aLine) + 1
              curChar = Mid(aLine, i, 1)
              If weHaveTabs Then
                  If curChar <> vbTab Then numAcctsStr = numAcctsStr & curChar
              Else
                  If curChar <> " " Then
                      numAcctsStr = numAcctsStr & curChar
                  Else
                      If Len(numAcctsStr) >= 2 Then Exit Do
                  End If
              End If
              i = i + 1
          Loop
          openingCountsStore = openingCountsStore & " " & numAcctsStr
      End If
  Loop
  fin.Close

  branchNames = Strings.Split(Trim(branchNamesStore))
  openingCounts = Strings.Split(Trim(openingCountsStore))
  Debug.Assert UBound(branchNames) = UBound(openingCounts)

  For Each par In aDoc.Paragraphs
      docLine = Mid(par.Range.Text, 1, Len(par.Range.Text) - 1)
      If docLine = "\begin{acc_opening_counts}" Then
          printingCounts = True
          GoTo nextItem
      ElseIf docLine = "\end{acc_opening_counts}" Then
          printingCounts = False
          GoTo nextItem
      End If
   
      typedNumber = False
      If printingCounts Then
          curBranch = Trim(par.Range.Words.First)
          par.Range.Select
          Selection.EndKey Unit:=wdLine
          For i = 0 To UBound(branchNames)
              If branchNames(i) = curBranch Then
                  typedNumber = True
                  Selection.TypeText Text:=" " & openingCounts(i)
                  GoTo nextItem
              End If
          Next i
          If typedNumber = False Then Selection.TypeText Text:=" 0"
      End If
nextItem:
  Next
End Sub

Sub importStats()
  Dim theDoc As Document
  Set theDoc = ActiveDocument

  Set fso = CreateObject("Scripting.FileSystemObject")
  monthFolder = "\\accounts\MB_REPORT 2004\YEAR2004\DECEMBER2004\"

  For i = 1 To 31
      If i < 10 Then
          fname = monthFolder + "Dhanmondi Branch  2004-12-0" & Str(i) & "\AC_OPEN_ALL"
          fname = Strings.Replace(fname, "0 ", "0")
          If fso.FileExists(fname) Then
              inputData fname, theDoc
          Else
              Debug.Print fname, "does not exist"
          End If
      Else
          fname = monthFolder + "Dhanmondi Branch  2004-12-" & i & "\AC_OPEN_ALL"
          If fso.FileExists(fname) Then
              inputData fname, theDoc
          Else
              Debug.Print fname, "does not exist"
          End If
      End If
  Next i
End Sub

May not look like much but was a bitch to write thanks to the lack of regular expressions in vanilla Office VBA. This was the first half. The second half was even harder because even more ill-defined -- almost no one's ever done it before.

The sparkline generator:
Microsoft Word Object ThisDocument

Private Sub Document_Close()
  myDocumentClose
End Sub

Private Sub Document_Open()
  myDocumentOpen
End Sub

Module NewMacros

Function sizeof(arr)
  sizeof = UBound(arr) - LBound(arr)
End Function

Function sort(arr As Variant, Optional SortAscending As Boolean = True)
  ' Chris Rae's VBA Code Archive - http://chrisrae.com/vba
  ' By Chris Rae, 19/5/99. My thanks to
  ' Will Rickards and Roemer Lievaart
  ' for some fixes.
  ToSort = arr
  Dim AnyChanges As Boolean
  Dim BubbleSort As Long
  Dim SwapFH As Variant
  Do
      AnyChanges = False
      For BubbleSort = LBound(ToSort) To UBound(ToSort) - 1
          If (ToSort(BubbleSort) > ToSort(BubbleSort + 1) And SortAscending) _
             Or (ToSort(BubbleSort) < ToSort(BubbleSort + 1) And Not SortAscending) Then
              ' These two need to be swapped
              SwapFH = ToSort(BubbleSort)
              ToSort(BubbleSort) = ToSort(BubbleSort + 1)
              ToSort(BubbleSort + 1) = SwapFH
              AnyChanges = True
          End If
      Next BubbleSort
  Loop Until Not AnyChanges
  sort = ToSort
End Function

Function arrayMin(theArr)
  Dim arr()
  ReDim arr(UBound(theArr))

  For i = LBound(theArr) To UBound(theArr)
      arr(i) = Val(theArr(i))
  Next i

  If sizeof(arr) = 1 Then
      arrayMin = arr(LBound(arr))
  ElseIf sizeof(arr) = 2 Then
      If arr(LBound(arr)) < arr(UBound(arr)) Then
          smaller = arr(LBound(arr))
      ElseIf arr(UBound(arr)) < arr(LBound(arr)) Then
          smaller = arr(UBound(arr))
      End If
      arrayMin = smaller
  Else
      sortedArr = sort(arr)
      arrayMin = sortedArr(LBound(sortedArr))
  End If
End Function

Function arrayMax(theArr)
  Dim arr()
  ReDim arr(UBound(theArr))

  For i = LBound(theArr) To UBound(theArr)
      arr(i) = Val(theArr(i))
  Next i

  If sizeof(arr) = 1 Then
      arrayMax = arr(LBound(arr))
  ElseIf sizeof(arr) = 2 Then
      If arr(LBound(arr)) < arr(UBound(arr)) Then
          bigger = arr(UBound(arr))
      ElseIf arr(UBound(arr)) < arr(LBound(arr)) Then
          bigger = arr(LBound(arr))
      End If
      arrayMax = bigger
  Else
      sortedArr = sort(arr)
      arrayMax = sortedArr(UBound(sortedArr))
  End If
End Function

Function scaleHeight(num, max, theHeight) As Double
  If max = 0 Then
      scaleHeight = 0
  Else
      scaleHeight = theHeight - (num / max) * theHeight
  End If
End Function

Function lineChart(aLine, theHeight, widthMul, showAvg, vertPos As Single, ByRef header, Optional ByVal scaleSame As Boolean, Optional scaleMax, Optional scaleMin)
  If Right(aLine, 1) = vbCr Then
      theLine = Left(aLine, Len(aLine) - 1)
  Else
      theLine = aLine
  End If
  theSeries = Split(theLine) ' Contains the header label
  Dim numSeries() ' Does not hold the label

  numNils = 0
  For counter = 1 To UBound(theSeries)
      If theSeries(counter) = "nil" Then numNils = numNils + 1
  Next counter

  ReDim numSeries(UBound(theSeries) - numNils - 1)
  For i = numNils + 1 To UBound(theSeries)
      numSeries(i - numNils - 1) = Val(theSeries(i))
  Next i

  If scaleSame Then
      min = scaleMin
      tempMax = scaleMax
  Else
      min = arrayMin(numSeries)
      tempMax = arrayMax(numSeries)
  End If
  max = tempMax - min

  For i = 0 To UBound(numSeries)
      tempNum = numSeries(i) - min
      numSeries(i) = tempNum
  Next

  If showAvg Then
      sum = 0
      For Each elem In numSeries
          sum = sum + elem
      Next
      avg = sum / UBound(numSeries)
      avgHeight = scaleHeight(avg, max, theHeight)
  End If

  With ActiveDocument.shapes.BuildFreeform(msoEditingAuto, (numNils * widthMul) + 100, scaleHeight(numSeries(0), max, theHeight) + vertPos)
      For i = 1 To UBound(numSeries)
          .AddNodes msoSegmentLine, msoEditingAuto, ((numNils + i) * widthMul) + 100, scaleHeight(numSeries(i), max, theHeight) + vertPos
      Next i
      freeformName = .ConvertToShape.Name
  End With
   
  With ActiveDocument.shapes.AddShape(msoShapeOval, (numNils + i - 1) * widthMul - 2 + 100, scaleHeight(numSeries(i - 1), max, theHeight) + vertPos - 2, 4, 4)
      .Fill.Visible = msoTrue
      .Fill.Solid
      .Fill.ForeColor.RGB = RGB(51, 102, 255)
      .Line.ForeColor.RGB = RGB(51, 102, 255)
      dotName = .Name
  End With
   
  With ActiveDocument.shapes.AddTextbox(msoTextOrientationHorizontal, (numNils + i - 1) * widthMul + 5 + 100, scaleHeight(numSeries(i - 1), max, theHeight) + vertPos - 7.5, 50, 15)
      .TextFrame.TextRange.Text = strings.Trim(Str(numSeries(i - 1) + min))
      .TextFrame.TextRange.Font.Size = 8
      .TextFrame.TextRange.Font.Color = RGB(51, 102, 255)
      .Fill.ForeColor.RGB = RGB(255, 255, 255)
      .Line.Visible = False
      .Fill.Transparency = 1#
      textBoxName = .Name
  End With

  If showAvg Then
      With ActiveDocument.shapes.AddLine(numNils * widthMul + 100, avgHeight + vertPos, (numNils + i - 1) * widthMul + 100, avgHeight + vertPos)
          .Line.ForeColor.RGB = RGB(153, 51, 0)
          .Line.DashStyle = msoLineRoundDot
          avgLineName = .Name
      End With
  End If

  header = theSeries(0)
  If showAvg Then
      retval = Array(freeformName, dotName, textBoxName, avgLineName)
  Else
      retval = Array(freeformName, dotName, textBoxName)
  End If

  For Each elem In retval
      Debug.Print elem
  Next
  lineChart = retval
End Function

Sub selectChart()
  Dim ctl As CommandBarComboBox
  Set ctl = CommandBars("Sparklines").Controls(2)
  If ctl.ListCount < 1 Then Exit Sub

  lineArr = Split(ctl.List(ctl.ListIndex), ":")

  theNames = Split(Trim(lineArr(1)), ",")
  Dim shapeNames() As Variant
  ReDim shapeNames(UBound(theNames))
  For i = 0 To UBound(shapeNames)
      shapeNames(i) = theNames(i)
  Next i

  ActiveDocument.shapes.Range(shapeNames).Select
End Sub

Sub deleteChart()
  Dim ctl As CommandBarComboBox
  Set ctl = CommandBars("Sparklines").Controls(2)
  If ctl.ListCount < 1 Then Exit Sub

  selectChart
  Selection.Delete
  ctl.RemoveItem ctl.ListIndex
End Sub

Sub moveChartRight()
  Dim ctl As CommandBarComboBox
  Set ctl = CommandBars("Sparklines").Controls(2)
  If ctl.ListCount < 1 Then Exit Sub

  selectChart
  Selection.MoveRight
End Sub

Sub moveChartLeft()
  Dim ctl As CommandBarComboBox
  Set ctl = CommandBars("Sparklines").Controls(2)
  If ctl.ListCount < 1 Then Exit Sub

  selectChart
  Selection.MoveLeft
End Sub

Sub refresh()
  myDocumentClose
  myDocumentOpen
End Sub

Sub myDocumentOpen()
  CommandBars.Add(Name:="Sparklines", Temporary:=False).Visible = True

  With CommandBars("Sparklines")
      With .Controls.Add(Type:=msoControlButton, Temporary:=False)
          .Caption = "Line Chart..."
          .Style = msoButtonCaption
          .OnAction = "lineChartGui"
      End With
      .Controls.Add Type:=msoControlDropdown
      With .Controls.Add(Type:=msoControlButton, Temporary:=False)
          .Caption = "Select"
          .Style = msoButtonCaption
          .OnAction = "selectChart"
          .Enabled = True
      End With
      With .Controls.Add(Type:=msoControlButton, Temporary:=False)
          .Caption = "Delete"
          .Style = msoButtonCaption
          .OnAction = "deleteChart"
          .Enabled = True
      End With
      With .Controls.Add(Type:=msoControlButton, Temporary:=False)
          .Caption = "Refresh"
          .Style = msoButtonCaption
          .OnAction = "refresh"
          .TooltipText = "Clears names of all charts from the list, whether charts are still in document or not"
          .Enabled = True
      End With
  End With
End Sub

Sub myDocumentClose()
  Dim sl As CommandBar

  On Error Resume Next
  Set sl = CommandBars("Sparklines")
  If sl Then sl.Delete
End Sub

Sub lineCharts(theHeight, widthMul, showAvg)
  noTb = False
  Dim sl As CommandBar
  On Error GoTo makeToolbar
  Set sl = CommandBars("Sparklines")

continueWithTb:
  howMany = Selection.Range.Paragraphs.Count
  If howMany < 1 Then End

  lines = Split(Selection.Range.Text, vbCr)
  theHeader = ""
  For i = 0 To howMany - 1
      theShapes = lineChart(lines(i), theHeight, widthMul, showAvg, 100 + i * (theHeight + 15), theHeader)
      shapesStrTemp = ""
      For Each elem In theShapes
          shapesStrTemp = shapesStrTemp & "," & elem
      Next
      shapesStr = Right(shapesStrTemp, Len(shapesStrTemp) - 1)
      sl.Controls(2).AddItem theHeader & ":" & shapesStr
  Next i

  Exit Sub
makeToolbar:
  myDocumentOpen
  GoTo continueWithTb
End Sub

Sub lineChartGui()
  frmLineChart.Show
End Sub

Yup, very complicated. Hopefully will become simpler and simpler in future iterations. Sometimes wonder why I don't just switch to automating Excel charts.

Sparklines: can't resist

When I started looking at ways to automate the graphing of the bank's accounts opening data, I originally started out with a 3-D line chart powered by a PivotTable. But have since realised that this is a perfect area of application for sparklines, Edward Tufte's `intense, simple, word-sized graphics'. For example, see above.

They're usually supposed to be surrounded by more context, but basically that is their size and general appearance.

Sparklines have so much potential in charting huge amounts of data; couldn't resist spending a lot of thought and time trying to figure out what would be the best way to implement them. First decided on plain HTML and CSS generated by Python, and spent a lot of time on it before decided it was too tedious because I had to get Python to generate each and every dot making up the lines. Python is very good, but after a while I realised I should use an environment which already provided vector-based drawing tools which could be automated.

The obvious choice turned out to be Microsoft Word, because of how common it is, especially here in Bangladesh. After some hacking, came up with the following code:

Const theHeight = 50
Const widthMul = 1

Function scaleHeight(num, max) As Double
   num = Val(num)
 
   scaleHeight = theHeight - (num / max) * theHeight
End Function

Sub genSl()
   Dim c As Shape ' Holds the canvases one by one
   min = 0
   max = 0

   Dim theArray()
   howMany = Selection.Range.Paragraphs.Count
   ReDim theArray(howMany - 1)
   Dim canvasNames()
   ReDim canvasNames(howMany - 1)
 
   For i = 0 To howMany - 1
       theArray(i) = Strings.Split(Selection.Range.Paragraphs(i + 1).Range.Text)
       For j = 1 To UBound(theArray(i))
           If Val(theArray(i)(j)) < min =" theArray(i)(j)"> max Then max = theArray(i)(j)
       Next j
   Next i
   max = max - min
 
   For i = 0 To howMany - 1
   ' For each paragraph in the selection a sparkline is drawn
       Set c = ActiveDocument.Shapes.AddCanvas(100, i * (theHeight + 20) + 200, widthMul * (UBound(theArray(i)) + 1) + 55, theHeight + 15)
       canvasNames(i) = c.Name
     
       With c.CanvasItems.BuildFreeform(msoEditingAuto, 0, scaleHeight(theArray(i)(1), max) + 7.5)
           For j = 2 To UBound(theArray(i))
               ' j starts from 1 because the first point was plotted in the BuildFreeform function
               .AddNodes msoSegmentLine, msoEditingAuto, j * widthMul, scaleHeight(theArray(i)(j), max) + 7.5
           Next j
           .ConvertToShape
       End With

       j = j - 1
       With c.CanvasItems.AddShape(msoShapeOval, j * widthMul - 2, scaleHeight(theArray(i)(j), max) + 7.5 - 2, 4, 4)
           .Fill.Visible = msoTrue
           .Fill.Solid
           .Fill.ForeColor.RGB = RGB(51, 102, 255)
           .Line.ForeColor.RGB = RGB(51, 102, 255)
       End With
     
       With c.CanvasItems.AddTextbox(msoTextOrientationHorizontal, j * widthMul + 5, scaleHeight(theArray(i)(j), max) + 7.5 - 7.5, 50, 15)
           .TextFrame.TextRange.Text = Strings.Trim(Str(theArray(i)(j)))
           .TextFrame.TextRange.Font.Size = 8
           .TextFrame.TextRange.Font.Color = RGB(51, 102, 255)
           .Fill.ForeColor.RGB = RGB(255, 255, 255)
           .Line.Visible = False
       End With
   Next i
 
   ActiveDocument.Shapes.Range(canvasNames).Select
End Sub

Sub showMarkers(n As Integer)
   pWidth = ActiveDocument.PageSetup.PageWidth
   pHeight = ActiveDocument.PageSetup.PageHeight
 
   Dim l As Shape
   For i = 1 To Int(pWidth / n)
       Set l = ActiveDocument.Shapes.AddLine(i * n, 0, i * n, 10)
       Set l = ActiveDocument.Shapes.AddLine(i * n, pHeight, i * n, pHeight - 10)
   Next i
   For i = 1 To Int(pHeight / n)
       Set l = ActiveDocument.Shapes.AddLine(0, i * n, 10, i * n)
       Set l = ActiveDocument.Shapes.AddLine(pWidth, i * n, pWidth - 10, i * n)
   Next i
End Sub

Sub doShowMarkers()
   Call showMarkers(10)
End Sub

If you're interested in using them, put them in some module in one of your documents templates (if in the Normal template, it will be available to all documents). Then put some data and numbers in the document itself, arranged in a certain way. The above sparklines were generated from the following data:

DSE 2 3 4 7 3 7 4 119 3
DSEGeneralIndex 749.11 768.03 795.05 763.7 752.91 792.56 874.57 870.46 874.22 842.36 845.07 848.41 807.6 806.92 750.84 787.94 791.7
DSE20Index 942.46 958.2 1004.56 963.88 920.73 973.88 1134.34 1094.45 1085.97 1052.47 1051.48 1054.89 1004.61 1021.5 948.27 964.13 964.32
RandomIndex 642.2 221.5 2

That is, each series is on its own paragraph (paragraphs not separated by blank lines), each item in the series separated from the other by a single space. To chart the data, select it all. If the selection contains a single data series, then a single sparkline will be drawn, and so on.

Need to work more on the code and especially on the GUI front-end. But for now it works OK.
Will upload it to a public server after working on it some more.

New ideas

As usual, haven't posted in a long time. Never found much to talk about, but nowadays I find myself looking at problems and inconveniences in my life, and others', and thinking of ways to solve them.

Example. With the abolishing of rickshaws from the main road near leading up to New Market and Nilkhet, the road in front of New Market has become more jammed than ever with parked cars and stationary rickshaws. Right now it is a two-way street, with two lanes on each side and a lane for parking cars on. A simple way to solve the jam would be to allow only cars on the side further away from NM, and only rickshaws on the side closer to it. Sure, cars would have to exit through the other, further side on their way out, but then, that's what they're there for.

Work

Started at One Bank in the beginning of December. Worked, or observed, my way through a lot of stuff but I've finally seen what to me is the most interesting part of it all: the raw data generated by the computer system of the bank's daily activities. These data are in the form of plain text files arranged into folders, essentially by date. They are just crying to be pulled in and processed programmatically by Excel or some such program. For example, there are daily data files about fixed deposits which mature on the day; and new accounts (including loans) which were opened on the previous day.

In the new accounts example, the information in each file (each day's report) includes a grouping of accounts by branch, count of new accounts in each branch, and detailed information on each account (one account per line). The way it is arranged makes it possible to parse it and pull out the most useful data -- for example, the count of new accounts opened in each branch. If one does this every day to keep current, one can graph the daily account opening activity for each branch, and what's more, put these graphs together into a combined `3-D' graph for ease of comparison. This gives, over time, a nice high-level view of account activity throughout the bank.

This is exactly what am now trying to do with Excel and a well-crafted macro at the bank. Have made some progress, and think the parsing bit is taken care of thanks to Excel's, well, excellent plain text file importing/parsing capabilities. But a lot of it is still left, including programmatically generating pivot tables and charts for new months. Should be quite a challenge. If they let me do this, even intermittently, it should make it very interesting at work. Don't know who it will really help, though, to be realistic. At this point it's just a shiny toy, a very high-level view which branch employees may not find ultimately useful and thus may lose interest in rapidly.

But still look forward to exploring more of the daily reports and perhaps even getting something useful out of them.

Evolution v intelligent design, notes from Slashdot

Kansas' education board has decided that its students should be taught about intelligent design, an alternative `theory' to evolution. Here's a posting from Slashdot (http://science.slashdot.org/science/05/11/08/2338233.shtml?tid=123&tid=14) which describes exactly how I feel about this:

Re:You are only hurting yourself you know.... (Score:4, Insightful)

by Decaff (42676) on Tuesday November 08, @10:15PM (#13985193)

Interesting comment--considering that they are teaching Intelligent Design alongside Evolutionary Theory. Your comment seems to indicate that, by teaching ONLY Evolution, that's how we develop Independent Thinking? Tell one side of a story? Somehow, that seems more like indoctrination to me.

You are missing the point. These classes are supposed to be science lessons, not philosophy or religion. There are plenty of alternatives ideas to evolution that can be discussed in biology classes, such as the ideas that fossils aren't old and the Earth was created recently. These areas are testable, and examining the data that suggests they are false can be highly educational - students learn about rock strata and radioactive dating.

Intelligent design is not testable. It is nothing more than a series of statements of incredulity - that because we don't yet understand everything about the evolution of life then there must have been intervention by a `designer'. This isn't science. Intelligent design might be science if there was some sort of valid consistent test for the existence of a designer, but there isn't. Also, because it is likely there there will always be some area of evolution or of biology that is not fully understood, there will always be some room for someone to say `that must be designed'. This means that Intelligent Design is never refutable; again, making it meaningless in the context of science.

Science teaching should include the idea that we are simply currently ignorant about some things. Coming up with untestable, irrefutable explanations to cover that ignorance is dishonest and should not be part of the process.

Imagine this sort of approach being used in other areas of science (e.g. `We don't yet fully understand the origin of comets, so aliens or gods must have made them') and the results are silly in the extreme.

Here's another that echoes it:

Re:You are only hurting yourself you know....(Score:4, Insightful)

by Flower (31351) on Tuesday November 08, @10:25PM (#13985290)

(http://slashdot.org/)

What independent thinking? ID certainly doesn't promote it. It provides the ultimate out in the search for truth. It's too hard right now to explain *this* so the obvious answer is God did it! (And don't even try to claim it is some ambiguous creator that spontaneously created the eye. The second some pagan asserts that it was the Goddess who made it happen you'll see every ID proponent in Kansas heading out to smite that heretic down.)

ID's greatest sin is that it closes doors to scientific research. If God miraciously intervened and created the eye then there is no reason to try to find an explanation. God did it so leave it alone and don't question it. Obviously if a million believers can't figure it out what could a scientist accomplish? And if this can be done in evolution then why can't it be done in other sciences? The creation of the universe is too complex to really comprehend so all this fluff about researching gravity really doesn't have to be done because we can just attribute the really interesting mysteries to God.

ID isn't science. It's the same old shit that pioneers in science had to fight against and be abused by centuries ago.

You have to love Slashdot: they put into words exactly how I feel about this stuff. I don't even have to raise a finger and type the stuff out, it's so perfect. Best of all, their sense of humour:

Hey Kansas! (Score:5, Funny)

by Anonymous Coward on Tuesday November 08, @09:30PM (#13984798)

We're becoming a laughingstock of not only the nation, but of the world, and I hate that

HAHAHAHAHAHAHAHAAHAHA!!!!!!!

-- The World

And also:

Re:Misleading headline (Score:4, Funny)

by c0d3h4x0r (604141) on Tuesday November 08, @10:17PM (#13985212)

(http://c0d3h4x0r.0catch.com/ | Last Journal: Friday November 04, @05:23PM)

2)It redefined the meaning of science. According to the new definition, science is no longer is limited to searching for natural explanations for natural phenomena.

Excellent! So now student `science' fair projects can be about... well, pretty much anything!

Richard Feynman must be turning in his grave.

Cleaned my room

I cleaned my room tonight. And I mean really cleaned, with mop and cleaning liquid. This is the first time I've ever done that, I kid you not. Room looks and feels cleaner than ever, but I know it won't last :-(

I'm managing to post now, after all this time, because only now have I gotten some time (well, stolen some time) to do it. It's been amazingly hectic for the last few weeks. The assignments have come and gone one after the other, and the way I am -- very bad at managing my time -- they pile up till the weekend before they're due, and then I start; of course by then it's guaranteed that I'll be brain-fried in a couple of days. But just enough, and just in time, to hand the damn thing in, then sink into lethargy.

Well, not actually lethargy. I've been snatching time to read Azarello's 100 Bullets whenever I can. I have upto issue 52, and it's amazing. For me it started off with a whimper, but it's gotten stronger and stronger. The big deal is that they keep the story intelligent, with plot surprises for me. And the characters just suck you in. I hate that. But I like and appreciate it very much. Plus Risso's art is a treat. I would be very surprised if he hasn't already drawn one of the Batman books.

Speaking of which, did I mention how much I love Lucifer? Mike Carey's Lucifer, I mean. It's so easy for me to identify with him. Damn. I have a problem with authority but will make my own rules as I go along. And yes, I hate hypocrisy ... but there's a line between playing by your own rules and being a hypocrite. Even if it's not there all the time. But that's what makes life so fun ... the back and forth between the law and lawlessness.

Anyway, back to the story. How did I end up cleaning my room? I have a reputation (at least among ... myself) for avoiding that kind of drudgery. Although I do like to think of myself as a glutton for punishment. Anyway. I was supposed to go watch a movie at 9 (pm), but my ISN meeting ... spilled over ... into that time, so I decided to skip it. Was hungry and didn't want to dine on popcorn watching the movie anyway. So. I came back home after dinner at Pink, the restaurant right outside the campus, and just decided to get it over with. The cleaning, I mean. I figured I wouldn't get the chance again soon.

Anyway, gotta go now. I should probably take my bath. At 1:30 in the morning, that makes me a very odd person. But then I've always been a night owl. Aren't all university students?

Harry Potter and the Half-Blood Prince

I bought the book the day it came out on Dhaka, July 16th (from what I remember, Rowling wanted to release it on the day of the summer solstice, but also at midnight. So that ended up being July 16th in Dhaka). It was a hartal, so believe it or not, there was practically no one else at the bookstore compared to the crowd I'd been expecting. It cost Tk. 1,475 in anyone was wondering.

I started reading it immediately after getting home. I had to stop to do stuff like packing and meeting family, because it was the last day of my holiday in Dhaka, and I was flying the next morning. But I read whenever I could, and into the night. Around 2 am, I finished. It was well worth it.

When I started reading it, something felt wrong near the beginning. But I couldn't put a finger on it. Yesterday I realised that it was that in the second chapter, Rowling tells the story located somewhere far away from Harry -- and we the readers see something that Harry doesn't, something she's never done before. Think about it. Every device that Rowling has ever introduced to show something that happens far away from Harry or far before Harry's time -- Harry's connection to Voldemort's mind, the `memory dishes', the Dementors' awakening of long-dormant memories in Harry's mind, the journal of Tom Riddle, among others -- let Harry see and hear what's going on, as well as us.

So for the first time ever, the story shifts away from Harry and onto Snape. What does this mean? I have no idea.

Then, throughout the book, Harry has this growing paranoia and obsession with Malfoy. They've always been enemies, but before this book, Harry never attached such an urgency to finding out what Malfoy was doing. And his intuition proved right in the end. But the irony is that Dumbledore was willing to put his own life on the line to turn Malfoy from his father's and Voldemort's side, and wouldn't lift a finger to stop him.

Reading the book, you get the feeling that Rowling has got the whole of these characters wrapped around her brain, so easily do their interactions come together. But also, Harry getting together with Ginny felt pretty forced to me. They had some little interaction before, nothing that came before had hinted at the `little monster' of jealous love in Harry for Ginny. With Ron and Hermione, for example, you knew it was coming from book 4 -- the Krum stuff. That was extended here into the kind of interaction I'd always imagined they'd have before they got together.

The climax of the book -- between Harry, Draco, Dumbledore and Snape -- also felt like the most natural possible buildup to the big finish. I couldn't have asked for anything better.

Snape's escape, and Harry's failure to prevent him, showed just how much Harry trails the older wizards in actual power levels, and just how lucky he's been upto now fighting the Death Eaters. I could feel Harry's frustration as Snape turned aside every single spell Harry threw at him, but left Harry unharmed, a prize the Dark Lord would claim for himself.

And the situation that the last chapter sets up -- it's beyond anything that has ever happened in Harry's world. Suddenly it's become a much bigger and scarier place, and no more shielding, no more Dumbledore, Sirius or even Snape to take some of the heat off Harry. The only protector-figure left for Harry now is Hagrid, which is kind of fitting -- there's a connection between them. But you have to wonder if Hagrid's days are numbered too.

But I digress. Everything is torn away from Harry -- Hogwarts forgotten, training to be an Auror put on the backburner, and friends, except for perhaps only Ron and Hermione, left behind. He finally has a quest, but it's a huge one -- a whopper which I'm not sure how Rowling is going to fit it into a single book 7, unless much of the work has already been done for Harry. We can only hope.

But whether or not that's true, it's been driven like a nail into Harry's head that he has to, to the exclusion of everything else, kill Voldemort. Nice and chilling.

Holidays end

I go back to Malaysia on the 17th, after a nice and relaxing vacation in which I've done everything the way I wanted to, but haven't done everything I've wanted to. Oh well, maybe next time.

But now, it's almost over, and that sucks. I've been watching movies, reading comics, downloading music and videos, and everything else without any regard to the time or how much sleep I get in the day. The only thing that I've seemed to lack is time -- it just flew by. I've stayed awake for 24 hours straight or more trying to do everything I wanted to do and still I was forced to go to bed, unsatisfied, at dawn. Only to wake up in the afternoon tired and bleary-eyed.

I have made sacrifices, e.g. my eyes. They're teary and bleary and a little bit red all the time, and oh so itchy. So itchy that I can get an almost orgasmic satisfaction by just rubbing them with the backs of my hands. Instead I force myself to pull at the skin around them a bit to satisfy the itch -- for a little while. If I rub my eyes my hands come away wet with a foul-smelling fluid that presumably comes from the backs of my eyeballs.

Today my semester results came out and I've done tolerably in all four of my core units -- haven't failed anything like I feared at times. Was a little surprised to see that I did best in the management unit -- a Distinction -- but then I guess it must be because of the second assignment; I worked hard on that one. The worst grade, a Pass, was in microeconomics. Guess I shouldn't be surprised, because no matter how much I like economics, I've always sucked at it.

Microsoft Office

Following my success at installing Internet Explorer and MSN Messenger on my Linux laptop, I realised I might as well install Microsoft Office and make a clean job of fizzling out my open source utopia. So I did it, yesterday. I installed Office 2000, the latest version that was known, as far as I knew, to work with Wine.

The installation did error out at first, but a quick check of a couple of Wine websites told me the answer to that problem. So I managed to complete the install -- Word, Excel and PowerPoint, possibly the three most used desktop applications in the Monash computer labs. Well, Visual Studio/Visual C++ is up there, but still.

Anyway, after I installed everything and made sure it worked -- and it does, more or less -- I started wondering how far Wine has come at running Office XP. So I did some Googling and wouldn't you know it, Office XP now works with Wine -- again, more or less. Damn. Still, at least Office 2K has all the features I really need in Word and Excel -- tables within tables, and PivotCharts.

Using Word and Excel, even stripped of all their Windows XP eye-candy freshness, I quickly appreciate just how good the user interface is. How well everything has been put together and just gets out of your way, compared to something like OpenOffice.org. Take, for example, the pull-down menus of their word processors. Word's menus are of a reasonable size; they don't stretch down almost to the bottom of the screen. In OpenOffice.org Writer, they do.

Note: I'm not dissing OpenOffice.org here. I really, really like some of its features, and if I could help it, I'd use it over MS Office any day. Features like the true integration -- it's really one program that changes its menus, toolbars, etc. depending on what type of document you're viewing; Stylist and Navigator panes which quickly let you manipulate and apply all kinds of styles, and move around documents quickly; the built-in PDF export; the Python programmability (I've written a very useful little word count tool for Writer in Python -- remind me to talk about it some other time); and of course, the fact that it's free and open source, works pretty much flawlessly with MS Office documents, and runs on the Big Three operating systems of today -- Windows, Unix and Mac OS X.

But where does that leave me with MS Office? The truth is I need Excel to conveniently do data analysis in my statistics units, and Word as a last resort to open files I might come across.

Updating a page in a browser

Say you know that a web page has recently been changed, but no matter how many times you click Refresh/Reload, the new version just refuses to come up. What do you do? Hold down Shift, then click. This will force the browser to go out and download the page again, instead of lazily telling you it has the latest version.

Quick update on Internet Explorer and Wine

Well, turns out I didn’t need Internet Explorer 5 at all. The reason the IE6 installer I downloaded from Microsoft was not working with Wine was that I had the latest Wine version installed, and it had a bug which prevented IE6’s installer from working. I downloaded and installed a slightly older version of Wine (20041019), and voila, Internet Explorer 6 SP1 was installed (I’m running it now).

I then managed to go to the MSN Messenger website and let IE install Messenger 7 for (which I’m also running now). So now, instead of having to use the E-Messenger website all the time, I can sign in with the Real Deal(TM). Yessss.

Details? Feeling lazy now, maybe I’ll post them later.

Various

The Darth Side

Ever wonder what thoughts swirled around in the evil Darth Vader’s head during the events of Star Wars 4–6? Well, here’s your chance to find out – this blog is the personal narrative of the Dark Lord himself! ‘The Darth Side: Memoirs of a Monster’, http://darthside.blogspot.com/. The classic post to read: ‘Does It Hurt When I Go Like This?’

Bruce Schneier

Check out ‘Schneier on Security’, http://www.schneier.com/blog/. He writes about an always-interesting topic, security, which inevitably leads to many diverse and interesting discussions with his readers on his blog – nowadays probably more about politics than about anything else. But it’s always an interesting take on the topic.

Internet Explorer on Linux

It’s ironic that I, a Firefox freak, am searching for a download of Internet Explorer 5. But it’s true. I am trying to install IE5, as insane as that sounds. And what’s more, I’m not trying to do it on my Windows machine, because I have none. I’m trying to do it on my Linux box, to get MSN Messenger to work on Linux.

For those of you not familiar with Linux, this is quite a trick, because it requires a lot of special tinkering before you can run common Windows programs like Internet Explorer and MSN Messenger. Things are going at snail’s pace now because I can’t find a download of IE5 anywhere on the ’net. Guess I’ll just have to keep searching.

Uni

University life never lets up. First there was the Monash Cultural Night, and that was so hectic it made us, the organisers, not so eager to see each other again for a while. Ironic.

Then there are all these damned assignments that have you on a choke-hold. They keep coming, one after the other. You keep putting them off, claiming to do ‘research’ and ‘I’ll start them real soon.’ But then a couple of days before the due date, you start doing the research and writing, and voila! Instant assignment. Meanwhile your brain and eyes are stir-fried.

On the up side, though, the assignments’ marks make up the total marks for each unit, so all the pressure is not put on the exam results.

Star Wars Movie and ...

I am so looking forward to the 19th. Episode 3 comes out, and all assignments for the semester are finally done and over with. Ah, the closure. Of course, that just means the exams are coming up in an all-too-few number of days, but that also means so is my summer vac. Ah, the mixed emotions.

Another (ir)religion post

Yesterday night I had the usual `discussion' with another guy about religion. He's a Muslim, and a pretty dogmatic one at that. `Discussing' the basic concepts of religion with someone like that gets very old, very fast. For example, yesterday, he was of the opinion that I'm something really bad is going to happen to me, and I'm on the `wrong path' because I don't have any religion. Here's a basic idea of the exchange that followed:

--

Him: Man, I'm telling you, you should start believing, because see, if you keep on going this way, sooner or later something really bad is going to happen to you.

Me: Maybe. But if does, then don't you think that's unfair? If someone came up to you and told you, `If you don't do what this book here says, you're going to be punished....', wouldn't that make you angry?

Him: You can't question these things....

.
.
.

Me: But why am I on the wrong path?

Him: Because you don't follow the Holy Book.

Me: But why is following the Holy Book the right path? You've been told that it is, by other people. But how do you know that's it so?

Him: You shouldn't question these things.

Me: OK man, if I can't question your beliefs, then let's make it fair and say that you can't question mine.

Him: That's fine with me, even though you're on the wrong path and will burn in hell forever....

Me: I think we had a deal...?

Him: Yeah, I'm just saying, because you really are going to get hurt really badly some time if you don't start believing.

.
.
.

Him: Man, if you don't believe, you're going to burn in hell forever, even if you never do anything bad in your life. Even [somebody] who has drunk and committed murder and adultery and everything will get into heaven if they ask for forgiveness right before they die. (Slightly hysterically) You'll burn in hell forever!

Me: But what if I live my life as an atheist ask for forgiveness right before I die?

Him: ...

--

And then there was another thing that we touched upon: the Quran. Obviously Muslims believe that it comes straight from God, and yesterday I was trying, very subtly, to show this guy, and a friend of his, how maybe Muhammad and the Arabs of that time could have written it.

--

Them: The Quran had to come from God, because how else could it contain all the stories of the prophets like Musa (Moses), Isa (Jesus), Ibrahim (Abraham) ... in such great detail?

Me: Maybe the Arabs of that time knew of those stories from the old books, like the old and new Testaments, the Torah, etc....

Them: But the Arabs of that time couldn't read!

Me: But they could have heard the stories from someone or the other. You have to agree that at that time Makkah was a big trading city, and all kinds of people -- Christians, Jews -- traded there?

Them: Yeah....

Me: Then they could have brought the Bible and the Torah along with them, and told the Makkans and other Arabs the old stories.

--

But that did get me thinking. Suppose that there is a God, and He wants to test us while we live on earth. Suppose He has given us free will and wants us to use it to make our condition better, to live on earth with dignity and respect for one another. Maybe the real test is to see whether we can discipline ourselves even without the carrot and stick of eternal reward/punishment.

To make things more interesting, and harder for us, He has given us lots of holy books with lots of instructions and admonitions, thereby killing two birds with one stone. A beautiful masterstroke: the books contain the moral codes and authority to set mankind on a moral path, but also enough threats and contradictions to confuse and scare us. A test to see if we can overcome the confusion and the fear of being by ourselves to get to the next level, whatever that is.

Look that the history of religion in our world through this theory: everything is perfectly balanced. The feeling of fellowship and happiness from following a higher moral power, together; and the bloodshed and strife caused by conflicting beliefs. The holy books and beliefs give us just the right amount of guidance, while at the same time literally putting the fear of God in us, and the seeds of our own confusion and misery.

`Jonathan Strange & Mr Norrell'

Just a few days before flying to Malaysia I bought and read, in three days, Susanna Clarke’s 800-page debut novel, Jonathan Strange & Mr Norrell (see Amazon.com). Where to begin? Well, let me start with the back cover blurb. Basically it talks about the two magicians who appear in England to revive English magic. It talks about Mr Norrell as a reclusive scholar-type with an aversion to hazardous forms of magic. And of Jonathan Strange as a wild magician almost on the brink, one who is ready to try anything to gain fame and power.

It couldn’t be more wrong.

WARNING: SPOILER

‘Two magicians will appear in England. One will fear me. The other will long to behold me.’ – prophecy by the Raven King, from back cover

Only someone who has read the book can understand how much the characters have been misinterpreted by the marketers. In the book, Mr Norrell has an aversion to any type of publicity and a dislike for any magic he didn’t personally sanction. Clarke writes of him destroying would-be magicians by forbidding them from performing magic, and censoring any writings on magic – even going as far as to magically vanish books of magic right out of people’s homes to keep the public from reading them.

Strange on the other hand is presented as a very likeable character, a gentleman to the bone. Here’s something I roughly remember from the book:

‘Wellington asked Strange, “Could a magician kill people with magic?”

‘Strange frowned. “I suppose a magician could. But a gentleman never would.” ’

Strange is blessed with an unparalleled talent for magic. He discovers magic as a profession quite by accident (seemingly). Unbelieving, he goes with the flow and discovers that things happen at his command that have not happened for hundreds of years. He comes into contact with, and works with, Norrell. Yet the world he enters is unexplored and disquieting, unlike Norrell’s world of ‘safe English magic’. Strange has premonitions of things going on that are just beyond his level of awareness.

Things continue. Both magicians grow in power and renown, Strange more so than Norrell, because of his willingness to aid his country’s war machine abroad. He practices magic on very large scales – rearranging the Spanish countryside.

After the war his wife dies in very mysterious and troubling circumstances. He is a broken man, although he tries not to show it very much. All he has left is magic. And the magic is growing stronger within him every day, as he immerses himself within it. He picks up more strange vibes, premonitions, insights. He goes to Venice and finds it, too, steeped in magic. In Venice Clarke describes one of the most memorable scenes I’ve ever read in a book. It involves trees, lots of trees. Read the book!

Strange discovers that his wife is not dead, but has been enchanted away by a suave, fun-loving and cruel fairy. After this, he throws all caution to the wind and distills all the knowledge he has gained in his years as a magician into a way to access the fairy country, Faerie. He uses madness, his own madness, as a tool to contact ‘the gentleman’, as the fairy is called. So desperate, and yet hanging on by a thread, he manages to save his wife, and at the same time, with Norrell’s help, revive English magic – as part of a spell worked long ago by the Raven King himself.

In the end, the eye of the beholder beholds the eye of the beheld. But you have to read the book to understand this!

He doesn’t emerge unscathed – the conflict leaves him literally in eternal night, which he and Norrell carry with them wherever they go. Clarke ends the book on an open note, as if indicating more adventures (and hopefully more books) to come.

One recurring theme I noticed in the book was the English’s unwillingness to accept magical causes for something, even though they know that magic is possible and even likely with Strange and Norrell having opened the doors to it. Strange himself is guilty of this before his wife dies. I think her death teaches him a lesson – if anything that could be explained at all by magic, it probably is magic. Cruel way to learn, and cruel lesson, but this is not an innocent book.

University, and some books

AFTER a long time of hither and thither, I've finally bought the ticket. I am definitely going to Malaysia to study in Monash University. The student pass they have sent me will see me through Dhaka's airport and on to Kuala Lumpur, where I will be given a visa.

I've finished, within the last few days, Jean P. Sasson's Princess and Daughters of Arabia. The former was excellent. I found the book, of all places, in my grandfather's bookshelf. I've known of it for a long time, but for some reason or the other always passed over it, looking for interesting books. But....

I really like learning about things which have always been shrouded in mystery, and the book felt like a frank confession of Saudi royal life behind the veil. `Sultana' and Sasson's voices both felt sincere, and I identified with the princess. I have very strong and conflicting ideas about freedom and laissez-faire and non-violence and the fair treatment of minorities -- among other things. I love to see them echoed in other people.

Daughters was not as original -- and understandably so. Still highly readable, though. I'll never tire of Sultana's opinions and thoughts, partly because they're so much my own. I'm waiting for my brother to buy the next in the series, Desert Royal. (I'm currently broke -- in Bangladesh taka, anyway.) Then again, I'll probably be in Malaysia by the time he gets around to it.

I'm currently reading The Da Vinci Code. Another excellent book. Symbology; hidden meanings in famous artworks; secret societies; radical theories about Christianity -- it's all there and it's exactly my kind of brainy thriller. I can't wait to finish it, but am loath to at the same time -- because a book like this is hard to find.

Reading the book, I couldn't help comparing it to another radical book I've read recently -- Anne Rice's Memnoch the Devil. That book was about the vampire Lestat's quest for redemption, but also about the struggle between God and the Devil, apparently called Memnoch by himself and Satan (among other names) by everyone else. In it, Memnoch explained to Lestat an alternate, and shocking, view of creation and mankind and Christianity's history.The parts all fit cleverly, and the wheel turns. It left me genuinely frightened (not from the religious repercussions -- I'm secular -- but from the general atmosphere created in the book. Yes, it was that good).

Another thing which I've been exploring is a certain similarity of themes between the Princess books and The Da Vinci Code. In the former, Sultana has the feeling that many problems in the world are caused by male domination over women, which is unnatural. In the latter, much is made of the natural duality and equality of man and woman, and the subversion of that natural order in the modern world, and the resulting pain and misery. Almost uncanny.

For those who died: we came from the ocean, and we will go back to it

`'Tis double death to drown in sight of shore.'
--
Shakespeare

THE mind boggles at the amount of death and suffering. The sheer scale of the disaster is unthinkable. The damage will take decades to repair. The blow will lay low the economies of the region and send it into an interrugnum of darkness and despair.

This is what floats through my mind right now, watching and reading about ten thousand new deaths with every new day and the rising difficulties facing the relief workers. Suddenly the Iraq insurgency, terrorist threats and Middle Eastern oil cartels seem like such distant concerns. Something has hit us here in Asia, and this something that could destroy this region and turn it into a hellpit of misery.

I look at satellite imagery of Indonesia before and after the tsunami and try to imagine what would happen if something like that hit here in Bangladesh. I can't; it's too hard.

I once read a Jungle Book story where Mowgli has the elephants of the jungle destroy his native village, which cast him out as a demon child and would have burnt his parents as witches. He said, `Let in the jungle, Hathi!' In the satellite pictures it looks like someone cried `Let in the sea!' and it was so. In the Jungle Books, there was nothing older or more powerful than the Law of the Jungle. Now it seems to me there is something older and more powerful: the Law of the Ocean.

And yet in the news they're always talking about what the UN is saying, what countries are forming coalitions, sending each other aid; the rich giving to the poor, and the poor sharing amongst themselves. Volunteers helping stranded tourists find their lost ones. All countries coming together in symphony and no jarring notes, no dissension. It's so amazing, so incredible to see this -- the countries acting as if they are part of one big nation, one nation-planet, governed by the common Law of Humanity.

Which is the stronger law? No doubt about it, the Law of the Ocean. We came from the ocean, and back to it we will go in the end.