Wednesday, March 28, 2012

Iterating Data Movement

Hi all

I am wanting to continuously monitor a source table throughout the day and as data becomes available, process it and insert it into one of a number of tables.

I have tried achieving this using a FOR LOOP and setting the halt condition such that it is not stisfiable. However, this has a couple of problems:

1) It runs in a tight loop and consequently degrades system performance enormously.

2) I can't get transactions to work. I would like each iteration of the loop to spawn a new transaction under which the tasks in the loop can run. Therefore, if one of the tasks fails during such an iteration, only the updates affected by that iteration are lost.

Ideally, I would like to be able to put a wait statement within the loop container so that it runs every couple of seconds. And would also like to implement transactions as described above.

All help is appreciated.

Jays :-)

I think you want a trigger on the table so you know when the data in that table is changed, and copy it accordingly.

|||

Hi Christian

I would like to avoid that if I can.

My package has custom logging that writes a set of audit records each time it is run. This would result in a LOT of these records being written throughout the day when one such set would do.

Thanks

Jays :-)

No comments:

Post a Comment