GO Transit notification service fail

Here’s an interesting observation in regards to the GO notification service. The activation link they send in the activation email has the following format:

http://enews-gotransit.com/activate.aspx?em=name@domain.blah

Does this mean Eve can hack together a script that will sign up all of her address book contacts to receive notifications for all possible notifications and activate it for them too? I think so. #fail

The Data Warehouse Problem

A post here offers to solve the following problem:

This one came in as a data warehouse problem in 1999. You have a history table of customer daily total purchases. The problem is to report just those  customers who decreased their purchase amounts on their most recent order placed with us.  We are trying to get an idea when people are saturated with whatever we are selling.  If their order level is holding steady we are happy with them.

Here is the table structure and a few rows of sample data:

CREATE TABLE DailySalesTotals (
    customer_id CHAR(10) NOT NULL,
    order_date DATE NOT NULL,
    order_amt DECIMAL(8,2) NOT NULL,
    PRIMARY KEY (customer_id, order_date)
);

INSERT INTO DailySalesTotals (customer_id, order_date, order_amt) VALUES ('Celko', '1999-11-28', 450.00);
INSERT INTO DailySalesTotals (customer_id, order_date, order_amt) VALUES ('Curly', '1999-11-25', 400.00);
INSERT INTO DailySalesTotals (customer_id, order_date, order_amt) VALUES ('Curly', '1999-11-26', 300.00);
INSERT INTO DailySalesTotals (customer_id, order_date, order_amt) VALUES ('Curly', '1999-11-27', 400.00);
INSERT INTO DailySalesTotals (customer_id, order_date, order_amt) VALUES ('Curly', '1999-11-28', 450.00);
INSERT INTO DailySalesTotals (customer_id, order_date, order_amt) VALUES ('Larry', '1999-11-25', 400.00);
INSERT INTO DailySalesTotals (customer_id, order_date, order_amt) VALUES ('Larry', '1999-11-26', 400.00);
INSERT INTO DailySalesTotals (customer_id, order_date, order_amt) VALUES ('Larry', '1999-11-27', 450.00);
INSERT INTO DailySalesTotals (customer_id, order_date, order_amt) VALUES ('Larry', '1999-11-28', 400.00);
INSERT INTO DailySalesTotals (customer_id, order_date, order_amt) VALUES ('Moe', '1999-11-25', 400.00);
INSERT INTO DailySalesTotals (customer_id, order_date, order_amt) VALUES ('Moe', '1999-11-26', 400.00);
INSERT INTO DailySalesTotals (customer_id, order_date, order_amt) VALUES ('Moe', '1999-11-27', 400.00);
INSERT INTO DailySalesTotals (customer_id, order_date, order_amt) VALUES ('Moe', '1999-11-28', 400.00);

The following is the original solution:

SELECT H1.customer_id, ' dropped purchase amount on ',
MAX(H1.order_date)
  FROM DailySalesTotals H1
 WHERE H1.order_amt
       < (SELECT H2.order_amt
            FROM DailySalesTotals H2
           WHERE H1.customer_id = H2.customer_id
             AND H2.order_date
                 = (SELECT MAX(order_date)
                      FROM DailySalesTotals H3
                     WHERE H1.customer_id = H3.customer_id
                      AND H1.order_date > H3.order_date
                   )
           )
  AND H1.order_date = (SELECT MAX( order_date)
                          FROM DailySalesTotals h4
                         WHERE h4.customer_id = H1.customer_id
                        )
 GROUP BY customer_id;

And here is what I came up with:

SELECT
    CUSTOMER_ID,
    ORDER_DATE
FROM (
    WITH RANKING AS (
        SELECT
            CUSTOMER_ID,
            ORDER_DATE,
            ORDER_AMT,
            RANK() OVER(PARTITION BY CUSTOMER_ID ORDER BY ORDER_DATE DESC) AS DATE_RANK
        FROM
            DailySalesTotals
    ) SELECT
        NEW.CUSTOMER_ID,
        NEW.ORDER_AMT - OLD.ORDER_AMT AS NET,
        NEW.ORDER_DATE
    FROM
        RANKING NEW,
        RANKING OLD
    WHERE
        NEW.CUSTOMER_ID = OLD.CUSTOMER_ID
        AND NEW.DATE_RANK = 1
        AND OLD.DATE_RANK = 2
)
WHERE
    NET < 0;

On a 130,000 rows of data, my query costs 1,895 vs. 361,770 of the original one. I am wondering if it's possible to do it more efficiently.

EDIT: and here is an absolutely beautiful ANSI-compliant query I found here that only costs 703:

SELECT
    CUSTOMER_ID, MAX(ORDER_DATE) AS ORDER_DATE
FROM
    (
        SELECT
            CUSTOMER_ID,
            ORDER_DATE,
            ORDER_AMT *
                CASE ROW_NUMBER() OVER(PARTITION BY customer_id ORDER BY order_date DESC)
                WHEN 1 THEN 1
                WHEN 2 THEN -1
                END AS total
        FROM
            DailySalesTotals
     )  T
GROUP BY
    CUSTOMER_ID
HAVING
    SUM(total) < 0;

EDIT2: A co-worker pointed out that using LAST/FIRST makes query even faster (cost 615):

SELECT
    CUSTOMER_ID,
    ORDER_DATE
FROM (
    WITH RANKING AS (
        SELECT
            CUSTOMER_ID,
            ORDER_DATE,
            ORDER_AMT,
            RANK() OVER(PARTITION BY CUSTOMER_ID ORDER BY ORDER_DATE DESC) AS DATE_RANK
        FROM
           DailySalesTotals
    ) SELECT
        CUSTOMER_ID,
        (
            MAX (ORDER_AMT) KEEP (DENSE_RANK LAST ORDER BY order_date) -
            MIN (ORDER_AMT) KEEP (DENSE_RANK FIRST ORDER BY order_date)
        ) AS NET,
        MAX(ORDER_DATE) AS ORDER_DATE
    FROM
        RANKING
    WHERE
        DATE_RANK < = 2
    GROUP BY CUSTOMER_ID
)
WHERE
    NET < 0;

A Stolen Life

Jaycee Dugard, who was kidnapped by a pair of sick people at the age of 11 and spent more than 18 years away from her family, could have chosen to write her memoir differently. She might have chosen to depict herself as a forever-a-victim type of person and this would have not been a surprise to anyone: a person who had been failed by the government, by more than one men she met in her life,  by the society itself, needs not a permission to feel like a victim. Yet I felt very little, if any, of this when reading Dugard’s memoir: A Stolen Life. Instead, her memoir painted:

A victim, who managed to stay sane and keep the strength in the darkest moments of her hopeless captivity:

I have feelings of hopelessness. I feel like nobody cares. This year has been extremely hard. [...] I’ve had a few bad dreams lately, too… nothing like before, though. One was about a serial killer coming and killing us all and nobody would ever know. Phillip says the angels give him terrible dreams, too, he says they make him feel dirty. Nancy has been having a terrible time, too. Lots of bad dreams that Phillip says the angels torture her with. Sometimes I don’t want to live on a planet that lets such horrible things happen. I will not give up, though.

A person, who knows how to avoid letting hate poisoning her soul:

[She says the following about her meeting with Nancy, one of the kidnappers, after Nancy was imprisoned - NP] She [Nancy - NP] said that she was scared when I walked in because she thought that I would hate her. I told her although I do not hate her because I do not want to pollute my body with hate, what she and Phillip did to me and my family was unforgivable.

A woman, who possesses an endless will to live and to rebuild the life forcefully taken away from her and also a woman who endured countless abuse but who is also able to go beyond her own healing and focus on spreading her love to others who have suffered abuse:

The JAYC Foundation evolved from a deep need to give back all that I was given. A pinecone was my last grip on freedom, so to me they represent what was stolen away from me. Now that I am free, they symbolize life and freedom. They are the seeds of new life and that is exactly what I have: “new life.” The pinecone is my reminder that life can always be restarted. But I know I can’t heal the world. To me the best place to at least start the healing process is within our own families. Given the right tools, even a family that has been torn apart by unimaginable circumstances can learn to build a new path together. The JAYC Foundation will be set up to support families willing to come together in a variety of situations and diverse circumstances. My hope is to provide counseling and housing for families and victims of abductions and exploitations during the crucial early days of reconnection. I will work to provide the same type of safe environment my family and I experienced during the early days.

To me, this is a truly sad yet beautiful book that teaches one to appreciate things s/he has, to be more aware of one’s environment, to hope that some of the greatest wounds endured may start healing one day.

Toastmasters Milestone: Competent Communicator

In June of this year, I’ve joined Comfortably Speaking Toastmasters where I’ve accidentally became a VP of Public Relations. A little less than two months later, in September, I became a member and an executive (again, purely accidentally) at SickKids Toastmasters, a lunch club that meets once a week near my place of work.

I am still not quite sure what made me come join Toastmasters that day in June, but I am thoroughly happy I did. I’ve met so many people that are great at public speaking that I have finally convinced myself that being an interesting and engaging speaker is actually something one could learn, with some effort.

A little less than 6 months later, I have completed the ten projects from the CC manual and I feel like I now know much more about my speaking abilities than I used to before I started. Toastmasters helped me to see my weak points and allowed me to develop the strong ones.

I am looking forward to working on the advanced manuals: I rushed through CC since I wanted to finish it before the end of year, but I am definitely taking the advanced speeches slower, trying to concentrate on the fine details of speech preparation and delivery.

If you have ever wondered what it’d be like to deliver speech and enjoy that, try Toastmasters and you will not regret.

BitNami – Trac on Windows

At my current job I don’t have the luxury of having a playbox that I can use for running my evil experiments. That also means that I have to take care of my own personal hosting needs using my laptop, which runs Windows 7.

If you ever had to bootstrap Apache2 under Cygwin, you would agree that the process is torturous. What’s more torturous than the setup itself (which, quite frankly, is nor more complicated under Cygwin than it is under Linux at times), are cryptic errors that seem to accompany Cygwin wherever it goes. Although I did manage to setup Trac stack under Cygwin, I could not really use it because apache would fail miserably every once in a while, complaining that it can’t fork() a new process, rendering the entire stack useless.

This is when I discovered BitNami. Guys and gals at BitNami describe their goals as:

The aim of BitNami is to simplify the deployment of web applications, such as wikis or blogs, in order to make them more accessible. There are a lot of high quality open source software packages that aren’t used as much as they could be because getting them up and running can be a complex process. We want to change that!

One of the delicious things they offer is a prepackaged trac/svn stack that is absolutely amazing. I got the entire thing running under Windows in less than 10 minutes. Oh, and the best part – it’s free.

Good movies are rare

I was re-watching some old (50s) Soviet comedies with my parents the other day. Every time I see movies from that time period, I find it astonishing how much film quality deteriorated in post-Soviet Russia when you look at these. My take on that is that the biggest selling point of older movies was their ability to provide a great story without substituting it for visual effects and millions of dollars worth of decorations. My parents say that it’s also because the number of movies coming out in the era of perestroika and before the boom of CGI was so little that people were willing to watch any, even completely tasteless, productions.

I don’t reject modern cinema, I just find it extremely hard to find movies that I would like be willing to watch over, and over again like I do with my absolute two favourites – Love and Doves and Office Romance.

If you don’t get a chance to watch the two above (I know, finding movie / subtitles may be a huge pain for those two), I would like to suggest a good contemporary one. This French movie is called “Un heureux événement (A Happy Event)” and I saw it this year at TIFF. The story revolves around a young mother, her husband, their new kid, and the crisis the entire family goes through. It’s a comedy that makes you think. Louise Bourgoin and the rest of the cast did an absolutely amazing job – I haven’t laughed so sincerely in a while and I highly recommend this one. As a bonus, Josiane Balasko is also in this movie (Le Hérisson).

Killing Lotus Notes – alternative to ZapNotes

I have recently learnt that the officially recommended way of recycling notes is:

cd C:\notes\

nsd -kill

Interestingly enough, worked better for me than ZapNotes.

Last day at IBM

This Friday was a lot of fun — final things taken care of, team lunch and warm goodbyes. The amount of sad feelings (because I was leaving) was canceled by the amount of happy feelings (because of the new opportunities and memories I’d have of all the amazing people that surrounded me for the past 1.5 years) and it felt very peaceful overall.

Ahead of me is a new job, another semester at UofT, and a few interesting projects. The number of interesting things happening around me grew a lot lately, and I am seriously consider starting writing weekly status reports just to keep myself on track on all of the things.

DB2 9.7 – setting up HADR with TSA failover

Note 1: IF YOU INTEND TO USE A TSA FAILOVER, YOU HAVE TO USE EITHER SHORT OR FULLY QUALIFIED DOMAIN NAMES EVERYWHERE CONSISTENTLY.
Note 2: Unless specified otherwise, run the following commands on both instances.
Note 3: For TSA parameters see:

http://publib.boulder.ibm.com/infocenter/db2luw/v9r7/topic/com.ibm.db2.luw.admin.mon.doc/doc/r0011471.html

http://publib.boulder.ibm.com/infocenter/db2luw/v9r7/topic/com.ibm.db2.luw.admin.config.doc/doc/r0052656.html

1. Create log directory (archived logs must be used for HADR):

mkdir /home/db2inst1/db2inst1/logs

2. Configure parameters on primary:

db2 update db cfg for SAMPLE using HADR_LOCAL_HOST hadrvm01 HADR_LOCAL_SVC 55555 HADR_REMOTE_HOST hadrvm02 HADR_REMOTE_SVC 55555 HADR_REMOTE_INST db2inst1 HADR_TIMEOUT 30 HADR_SYNCMODE SYNC HADR_PEER_WINDOW 60 logarchmeth1 disk:/home/db2inst1/db2inst1/logs LOGINDEXBUILD ON

3. Backup database on primary:

db2 backup db sample

4. Copy backup file and restore database on standby:

db2 restore db sample

5. Configure parameters on standby:

db2 update db cfg for SAMPLE using HADR_LOCAL_HOST hadrvm02 HADR_LOCAL_SVC 55555 HADR_REMOTE_HOST hadrvm01 HADR_REMOTE_SVC 55555 HADR_REMOTE_INST db2inst1 HADR_TIMEOUT 30 HADR_SYNCMODE SYNC HADR_PEER_WINDOW 60 logarchmeth1 disk:/home/db2inst1/db2inst1/logs LOGINDEXBUILD ON

6. Activate database on standby:

db2 activate db sample

If you receive a message that says that the standby database is already active, then activate the primary database, then start it for HADR as primary.

7. Start HADR on standby:

db2 start hadr on db sample as standby

8. Start HADR on primary:

db2 start hadr on db sample as primary

9. Test takeover on standby:

db2 takeover hadr on db sample

Some useful commands to check config:
db2 get db cfg for sample | grep HADR
db2pd -hadr -db sample

10. Clean up all rsct resources:

/usr/sbin/rsct/install/bin/recfgct -s

11. Check that machines get their time from the same NTP server, if they don’t:
On the NTP client machine, disable old servers and add:

server some.ntpd.server.hostname

to

/etc/ntp.conf

and then do:

service ntp restart

Should be able to get time and date from there.

13. Add these to

~/sqllib/db2nodes.cfg

On primary:

0 primaryhostname

On standby:

0 standbyhostname

13. Prepare nodes (both machines):

preprpnode server01 server02

14. Set up alternate servers:

db2 update alternate server for database SAMPLE using hostname hadrvm02 port 50001

db2 update alternate server for database SAMPLE using hostname hadrvm01 port 50001

15. Run:

/opt/ibm/db2/V9.7/install/tsamp/db2cptsa

16. Run db2haicu, first on standby then on primary. You want to create a public network for the interface cards, other steps are pretty intuitive.

Should be ready to go :)

locate vs. find: which one to use?

Many users don’t realize that searching files by filename can often be done more efficiently with locate than with find. The difference between the two is that locate uses a periodically updated filenames database while find searches hard drive on the fly. Generally speaking, an up-to-date database maintained by locate will yield the results faster than find.

Rules of thumb to using both are:

  • use locate whenever you are looking for older files such as configuration files, or something that has been on your hard drive for around a week or so;
  • use find whenever locate did not yield results :) – but also if looking for newer files (on my Ubuntu machine updatedb runs on daily cron schedule), or when there is a need need to search by complex attributes (such as modification timestamps), or when there is a need to use -exec in tandem with the search operation.

protip: even though locate database is brought up-to-date automatically, if you want to do it manually, simply run

sudo updatedb

.

Next Page »