Friday, January 29, 2010

Boxbe | GoodBye Email Overload!!!

This goes back to my other post a li'l while ago that covered some cool, must-use WEB2.0 sites if you want to really organize your life on the internet, be it social n/w, coding, searching, bookmarks or for that matter every damn thing you hook on to that wi-fi for (and I didn't say that you're floating on your neighbor's air-waves, got-ya!)

How many of us are now over those over-zealous spam-filters that Gmail!, Yahoo!, MSN and AOL! boast of so much? I'd say we rather take it for granted that spam is a part of life if you want to do email and buddy we can't help but sometimes go and sift thru those unwanted emails to actually see some of the important ones being junked too!!!

Well, this is Internet age; the number of innovators and thinkers grow much faster than the Moore's law would have predicted for the growth in number of transmitters on those Intel nano-scale chips...help isn't far away. Think of ways to make life better and you bet there will be someone who's gone a step further and actually done it and published it on internet for free!

Welcome to Boxbe! I won't say much here... would rather let the embedded video demo it to you.

What is Boxbe? from Randy Stewart on Vimeo.


Some highlights:
1) filters and prioritizes our email to reduce email overload.
2) Gives you an organized inbox, making it faster and easier for us to view, answer and send email.
3) It's free and it integrates directly with Yahoo! Mail, Gmail, Google Apps and AOL Mail.
If you don't use these, Boxbe offers a public forwarding address.

So, try this one out and say goodbye to multiple email logins and track and organize important conversations using this service.

Tuesday, January 5, 2010

LLVM | compile, JIT, do run-time optimizations on C/C++ and generate byte-code

Woah!!! This is cool.

We've all known the concept of VM's and the JVM that is being virtually used in every intelligent machine, even the thinnest of a computing client (cell-phones, e-gadgets etc.)

But, we also know that no-matter how much the JVM has evolved thus far... it's a virtual machine and can still not give you the performance of the real machine as close as possible (the in-famous performance debate: C/C++ compiled code Vs. Java byte-code)

Well, here comes the LLVM... Low Level Virtual Machine, a bit more closer to the platform and one that has benefits like:
1) effective optimization at compile time, link-time (particularly interprocedural), run-time and offline transformations (i.e., after software is installed)
2) sophisticated transformations on object code, while remaining light-weight enough to be attached to the executable.
3) static back-ends for many popular (and legacy) architectures, a back-end which emits portable C code, and a Just-In-Time compilers for several architectures
4) does not require garbage collection or run-time code generation (It makes a great static compiler).

To give it a try, write some C/C++ code and then compile it from within the browser.
Observe the LLVM re-generated code, how C/C++ constructs map to LLVM and run some LLVM optimizations.

Tuesday, October 20, 2009

Linux power shell commands and tricks: Part I

A lot comes bundled with Linux OS these days (thanks to the FOSS revolution) and we've zillions of utils and UI's to choose from to adapt to the Linux way of life in software development!

Still, Linux encourages the UNIX thought process and the philosophy of DIY (do-it-yourself)...
A power user (developer, build engineer, admin) would programmatically configure (read: automate) his environment and do more with the console using the plain old Linux "shell"

This series (to continue in parts) is on those power shell utilities (command combinations, scripts, performance tuning tweaks) that would help a user do more with less!!!

To start with, here I post a collection of such things that I've found useful:

1) A script (collection of commands taken from Scott Croft's Oracle Tuning on RHEL tweaks) to make your own network-configuration file (using static IPv4 address).

Note:
a) Need root level or sudo access to be able to run this script
b) Tested on RHEL and Fedora.

Takeaway: you'll have a Linux machine configured to communicate over an IPv4 network, with a static IP configuration.

You can copy-paste the below commands in an ASCII text file (with a .sh extension) and create a re-usable script or you can run the commands step by step:

HOSTNAME=$(grep HOSTNAME /etc/sysconfig/network|awk -F= '{print $2}')
hostname $HOSTNAME
GATEWAY=$(ip route list |grep default |awk '{print $3}')
echo "GATEWAY=$GATEWAY" >> /etc/sysconfig/network
DEFDEV=$(ip route list|grep default|awk '{FS=" "; print $5}')
IPADDR=$(ip addr show $DEFDEV |grep inet |grep -v inet6|awk '{print $2}'|awk -F/ '{print $1}')
echo "IPADDR=$IPADDR" >> /etc/sysconfig/network-scripts/ifcfg-$DEFDEV
sed -i 's/dhcp/static/' /etc/sysconfig/network-scripts/ifcfg-$DEFDEV
BCAST=$(ip addr show eth0 |grep inet |grep -v inet6|awk '{print $4}')
echo "BROADCAST=$BCAST" >> /etc/sysconfig/network-scripts/ifcfg-$DEFDEV
echo "NETMASK=255.255.255.0" >> /etc/sysconfig/network-scripts/ifcfg-$DEFDEV

2) Usage of "xargs", "exec" and pipe to do-more-with-less:

a) To add a new extension to all files in a dir:
ls | xargs -t -i mv {} {}.old

xargs reads each item from the ls ouput and executes the mv command.
The -i option tells xargs to replace {} with the name of each item. The -t option instructs xargs to print the command before executing it.

b) To rename a subset of files, specify the file names with the ls command:
ls *.log | xargs -t -i mv {} {}.old

c) To add a current timestamp extension:
ls *.log | xargs -t -i mv {} {}.`date +%F-%H:%M:%S`

The extension will look like ".2009-10-20-19:37:16"

d) To rename just the extension of files:
rename .log .log_archive.`date +%F-%H:%M:%S` *

This command replaces the first occurrence of .log in the name
by .log_archive.`date +%F-%H:%M:%S`

e) Find out exactly which files are eating up a big chunk of your disk space:
To find all files over 10,000KB (10MB+) in size and display their names, along with size,

find / -size +10000k | xargs -i ls -lh {}
or
find / -type f -size +10000k -exec ls -lh {} \; | awk '{ print $8 ": " $5 }'
or, in some other Linux variants:
find / -type f -size +10000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'

e.g.
# find /var/ -size +10000k -exec ls -lh {} \; | awk '{ print $8 ": " $5 }'
/var/lib/rpm/Packages: 69M
/var/lib/rpm/Basenames: 11M
/var/lib/rpm/Filedigests: 11M
/var/cache/yum/updates/filelists.xml.gz.sqlite: 57M
/var/cache/yum/updates/primary.xml.gz.sqlite: 21M
/var/cache/yum/fedora/filelists.xml.gz.sqlite: 14M
/var/cache/yum/everything/filelists.xml.gz.sqlite: 59M
/var/cache/yum/everything/primary.xml.gz.sqlite: 29M

...topics coming up next in Part II: memory, process, performance tuning and more!

Tuesday, October 6, 2009

Startup: Inventing Demand, chicken and egg problem

This one talks about business... captures how Steve was able to solve the Chicken n egg dilemma and launch his startup Rentoid that has picked up pretty well and is known as the ebay of Rent.

An interesting read about how Steve invented demand and created such an innovative biz.
Rentoid had a classic chicken and egg problem when it first got launched. People wont list items until we have willing customers waiting to rent their stuff. Conversely, people couldn’t rent things until people willing to rent their items put them up for rent. It’s a bit like asking two people who don’t know each other to fall in love. To solve this problem I decided to ‘Invent Demand’. This is how I did it.

I went out and got myself a copy of the Harvey Norman and all the major department store catalogs. Scan through and them and picked off what items I thought would be suitable to rent. For the purposes of rentoid that meant items that were ‘hot‘ in market (their placement in the catalogue was proof enough of that), items which had a purchase value of over at least $200, and had a low likelyhood of damage. I then proceeded to gather photos of the specific items off Google images and listed each of them on rentoid. The rental prices I placed at 5% of item value for a week, and 10% of item value for a month. The bond I made 50% of the cost. I made sure I listed items from varying categories. I did it in 3 suburbs across Melbourne (North, West & East). The listings also said ‘as new, never used’ – how true. It also assisted with our SEO because people do ‘item & location’ specific searches.
...
When people rented the items, I went out and bought them, first hunting for the lowest price on line. Then rented it to the new rentoid member in good faith and gave them an exceptional user experience.

After the rental I sold the item on ebay for around about 80% of the retail price. I pretty much re-couped my costs doing this.
Seemingly a little deceptive way to catch-the-fish, but is it really? People got what they wanted from the site and the process for both the renter and the rentee was completely transparent.

Customer Focus: Now that it's an established ebay-of-rent... look at what you get from the rentoid team when you email them:
Thanks for your curiosity!
We don't get nearly as many questions as you might imagine at rentoid. We reckon it's because our site is pretty easy to use and hasn't got a million confusing things on every page, or maybe people think we won't answer the query and some computer will.
Sure it's an automated response this time, but a real, living, breathing, person who hasn't nearly had enough sunlight will respond to your question within a day or so. We figure if you cared enough to ask something, we should care enough to respond personally.
If your question is super-urgent give Steve a call on [private number] or Luke on [private number].
Cheers,
The Rentoid team.
Steve figured a brilliant new way to build demand and momentum for his start up. A simple belief worked for him: an action creates a reaction and often people simply liking an idea isn’t enough to give a thumbs-up to a startup.
You got to take that idea forward, do a due diligence, break the rules.... beat the Catch-22 situation, create demand!!!

Friday, September 25, 2009

Cygwin on WinXP/Vista as an X Client

Most of us know cygwin as one of the best/easy Linux-like emulation on WIN/x86 platform. Cygwin also offers a graphical UNIX/LINUX client terminal (unlike putty that's just a character console).
To install/configure cygwin refer Shuva's blog on cygwin.

We can "ssh" to any X machine and open up a full fledged X console (KDE, GNOME etc. for Linux) in 3 steps:

1) From within the cygwin bash terminal, run:
$> x&
$> wmaker&
This will open up the cygwin GUI console that offers many utils as an X Client, including the XTerm.

2) Open Xterm on the wmaker console (right click and select Xterm)

3) On the XTerm command prompt run:
xhost +

4) login to the X host (the UNIX/Linux host to which you want to connect)
e.g.: "ssh username@hostIP"

5) Now that you're logged in to the UNIX/Linux machine, export its Display to your Windows machine IP (where you're running Cygwin) as:
$> export DISPLAY='X.X.X.X:0.0'
where X.X.X.X is your Windows machine IP (the character after : is "zero")

6) Start any X console, like:
$> startkde&
or
$> startX

... and voila!!! You can now see the GUI from your UNIX/LINUX host exported to your Windows client and running within the Cygwin wmaker terminal window. You may even export any other GUI utilities (like web-browser: firefox or email-client: evolution) and run/control them from your WIN client.

All said and done, this is a great, free and easy way to RDP to your UNIX/LINUX GUI console from your WIN box. But, things can go wrong sometimes with Cygwin.
Couple of usually faced issues are described below:

1) When you try to open Xterm window in wmaker (or try any other wmaker util), it doesn't open up and gives an error:
<<<
440 [main] wmaker: 1432 bash: fork_copy: linked dll data/bss pass 0 failed
>0x542000..
>0x5427F0, done 0, windows pid 1960, Win32 error 487
cannot fork a new process: Resource temporarily unavailable
fork failed: resource temporarily unavailable
>>>

Solution:
On your cygwin bash shell:

1) Check if you already have "rebase" package installed (/usr/bin/rebase and /usr/bin/rebaseall)

2) If yes, run the command:
$> /usr/bin/rebaseall
Now, try to open an XTerm window in wmaker and you should be good to go!

Note: Cygwin documents the need for a rebase/rebaseall for some packages/upgrades (like /usr/share/doc/cygwin/python-*.README)

Cygwin FAQ says:
Whether a rebase is required depends on several factors -

1) which packages you've selected (during an install or upgrade)

2) whether those packages have been compiled with --enable-auto-image-base.
The rebase is only required for certain packages, particularly
those that use dynamically loaded modules such as python or perl.

But, the fact is that running "rebaseall" does solve the above issue!

2) Cygwin bash shell (terminal) in WIN Vista doesn't open up fine; won't let you run basic commands; Permission Denied issues!

Solution:
The workaround is to run the Cygwin shell as Admin (before that, activate the real Administrator account)

a) Run Vista command prompt:
cmd

b) execute the following commands:
cd c:/cygwin/bin (or wherever cygwin is installed)
runas /user:Administrator bash

This would start cygwin bash shell with the Administrator account.
To setup the environment for Admin user properly, copy the skeleton files to admin's home folder /home/Administrator/:
execute the following command from the cygwin bash prompt:
. /etc/profile

While Cygwin, being a traditional one out there, works for most of us there are X clients like Xming that provide better GUI's and are compatible w/o tweaks with most WIN verrsions.

Monday, July 6, 2009

World E-Book fair is ON all thru-out this JULY

Folks!
What are we waiting for?
This is it, a collection of ~2 Million titles on nearly all the subjects/topics we know about (..and many that we don't)!!!

There are titles that are difficult to get from the shelf, there are ones that aren't affordable and there are some you'd just like to move your eye-balls thru w/o owning them.

So download/share/read e-books, mobile-books, e-text for your pdf-reader-gadget and
much more.
Visit the World-E-Book Fair, now!

Wednesday, May 21, 2008

The happening world of web2.0 services

Web2.0 seems to be the buzz around for many new services mushrooming all around the web (on a daily basis). These cater to the needs of people volunteering to fill the gap for things that range from bare necessities in a web-enabled PC user's life to just plain what if's...
Here, I've just put some of those websites I've hit (thru search or referrals or ads) recently.
EyeJot: what could be the next in emailing! Don't tell me all that Google, Yahoo and M$ have to offer with integrated search and schedulers and what not!
Well, Eyejot is different. A client-free online video messaging platform for personal and business communications.
Offers: The ability to create and receive video messages in a self-contained, spam-free environment.
Client-Free: With no client to install, start using Eyejot immediately with any browser, on any platform.

Yugma: If you're a MS LiveMeeting or WebEx user and still want more, then Yugma is the one killer Web2.0 app for ya.
It's free, and you can use it for Desktop Sharing, Teleconferencing, Presentations, Shared File Space and Session Recording.
Share your computer screen in real-time so that everyone sees what you see - regardless of the application, software or operating system you are using. With Yugma, you can accomplish any task you can in a traditional meeting.

Box: Ever thought how storage has become so important for us that now we need it on the fly, wherever we go and all the time. My laptop or an external drive can promise me some but not all the features that a web app dedicated to storage and collaboration gives us and yet again, free ;)
Box.net is an online Web storage provider. Users can store 1GB of whatever they want.
Pro Features: It provides slick-looking file-sharing widgets. Users get instant previews of images, music, and text documents. The files reside in your Box.net storage area, and as the widget owner you can even upload files through the widget.
Box.net has also launched a development platform for 3rd party sites and services to tap into it's file systems to run applications right within the file browser. Users can add these services to their Box.net accounts free of charge, and tap into them by right-clicking on files. There are more in this category out there like sugarsync that provides more sought after features like Multi computer data (music, text, video and wat not) sync and Automatic online backup (but it's a paid service app)... And the options are growing ;)

Google Reader: One of the greatest web2.0 hits from Google's stable of online apps, this one has simplified and popularized the single-window concept of getting abreast with information. Not just that most of the websites now have an RSS/Feedburner option for you to get the latest w/o re-visiting the site. With this app, users can subscribe to as many RSS feeds as they want, then browse them in a lean and simplistic two-pane story browser that feels a little bit like Gmail. Other than photos and text, Google Reader can display embedded video clips from several popular services.
It provides you features like: sharing the RSS feeds/stories with others. It even integrates your list of shared stories with buddies on Google Talk.
Works on any browser, versions available for mobile phones, the iPhone and the Nintendo Wii. Google offers iGoogle, a personalized page that can set up an RSS startpage for you and your colleagues. Netvibes is another one that offers similar features.

Mint: Don't worry, there has to be a web app for financial management services and that's where this one fits in, free.
Pro features: Interfaces directly with your bank and credit card companies to keep your accounts up to date. Not just a budgeting tool, but also a way to keep track of accounts w/o going to a bunch of different sites.
E-mail or mobile alerts. Caters to alerts like accounts dropping to critical levels, bills due, or when large purchases are made on credit cards you're tracking. Mint makes money for itself, and for you, by analyzing your spending habits and your accounts and recommending offers that will save you money. Got a high-interest credit card? Spending too much on mobile phone? Mint's advertising network will match offers from its partners to your particular situation.
Although this works only for people managing their stuff in the US currency it won't take much time for other such web apps (do I hear moneycontrol) to provide similar or better features for your localized needs.

As I write this review, there are hundreds of new web2.0 apps coming up to enable you with essential and good-to-have tools for productivity, entertainment and information search and analysis. Explore some of the best rated ones too.

Thursday, March 20, 2008

Do you use GoogleCodeSearch?

Recently one of my team mates was looking for a library call or some implementation to be able to get the subnet mask given a CIDR notation like 172.xx.xx.xx/16. There's a command "ipcalc" that can show you the subnet and other info given a CIDR notation, but we wanted to see how its implemented. With the kernel src code RPM not installed on that machine, we thought of doing a search for it's implementation on Google. It turned out that Google (which seems to be the parent of all searches) isn't as good in doing these type of code searches as it's more specific child Google Code Search

Although I've used Google Code search for finding out algos, implementations, generic API's for some typical scenarios and areas of interests in development, but it works much faster than tracing it locally for Open source code, even the kernel code and other system level implementation like in the above case.
What else, you get some really cool re-implementation of Linux code; some with improvements and customizations that can be useful.
The search results that we got also displayed implementations for some important related commands like ifconfig, iproute etc.

But then there's lot more to GoogleCode Search than searching just linux source code.
Like if you need to put your application or IDS/IPS firmware thru a burst traffic load then you can just search for a packet generator and voila!







For similar code searches I tried koders but it stands a bit low as can be seen by the search results (and interface) for the same query. Clearly, the interface as well as the results are not that intuitive and the search doesn't add anything to what you're looking for, say for e.g: more similar implementations or some links that provide more information on the searched item.

The one that gives a close fight to GoogleCodeSearch is Krugle.

With a search result providing different levels of implementations, modules to look for on this topic and one that also offers technical help, white papers and other related links on the searched context, this one sure is a better place to code-search. To promote it further, these guys offer an enterprise version that can sit on your desktop and provide code-searches both for internal and open source indexed code.

Other code search engine that can be referred:
Oreilly, Ucodit

I found another review for these search engines at: 5-great-code-search-engines

Code search can be fun too!

Friday, February 8, 2008

Tweak | How to optimize FireFox to use less Memory

I had been facing a lot of issues lately with Firefox when I've multiple FF windows and tabs open. Usually I'd have atleast 4 FF windows: Markets, Code/Tech Search, Reader/Blogs, Misc opened at my Desktop, each having half a dozen tabs.
With my belief that FF is less buggy (or so you'd say I'm biased towards open source browsers) and has less mem leaks than it's in-famous counterpart IE... I've been living pretty happily with this browser for some time until recently when I started facing heavy memory usage issues (maybe it's normal but I still wanted to squeeze it down when FF is resting at the task bar). Googled out one such solution:
Courtesy: This web-link

Steps:
Note: Just check your FF memory usage from task-manager for the no. of win/tabs open(firefox.exe)
1) On the website url toolbar type about:config
2) A page full of words will show up. Right-click anywhere and choose New -> Boolean
3) For the name input type "config.trim_on_minimize"
4) Select True
5) Restart FireFox.
6) Ctrl + alt + del (this opens your task manager)
7) Go to process and choose your Firefox process (usually called firefox.exe)
8) Look at the Memory Usage Column (average is 20,000-40,000, for me it was 100,000 for two windows having 2 tabs each, other things could also account for heavy mem usage; one being lot of script based, data-intensive, client-side calc, auto-refresh websites)
9) Now minimise firefox and you will see the memory usage to be less than 10,000 (for me it came down to 45,000).

Normally, the memory usage is bound to go down for most apps with Win-key+D option when you put everything un-used back into the task bar. But, if things still doesn't go better with FF, I believe this tweak is for us to use.

Just a follow-up: To have more such tweaks done in FF do visit this link

Wednesday, February 6, 2008

Just got hooked on to blogging? Register your URL to search engines for free!

And this isn't just it... There's a significant lot more we need to do when it comes to getting attention in this wide world of blogs (over a million blog-posts happening everyday)
Being a novice blogger myself, I won't cover all that you need to seek attention for your blog. Surely though, you can at least make the search engines to throw-up your blog URLs when a relevant keyword is searched for.

Here's what we can do:
1) Add your URL to Google using at: http://www.google.com/addurl/
Most of the search results of smaller/niche search engines are fed by crawlers/bots from Google.
So, adding your blog's URL to Google can not only make them appear in Google's search results (localized country wise) but also in AOL, Euroseek, Netscape and Earthlink.

And that's not it. Google helps you in getting more specific information about your website that can help your attention seeking efforts and tell you about page-hits, links to your page, broken-links, statistics etc. So, after registering the URL at Google, you're re-directed to Google Webmaster Tools.

Google Crawler does some sample searches for your site and provides indexing stats. Lists out various posts on your website, provides information on webpages that link to your website and a list of sites related to your site's name/content.
There are other options like Crawl stats and subscriber stats that offer information on page-hits and other link statistics.


Check-out a whole bunch of tools offered by Google to enhance your blog's visibility and hits on the internet, increase traffic to your site and connect you with interested visitors.


2) Yahoo! offers free website submission as well as paid Y!directory option. Do it at http://submit.search.yahoo.com/

3) MSN/Live is the third most popular search engine. The link to add your website in it's search index is http://search.msn.com/docs/submit.aspx

4) Gigablast: Add your website to Gigablast at: http://gigablast.com/addurl

5) Alexa: Add your website to Alexa, using: http://www.alexa.com/site/help/webmasters

6) AOL: If you are registered with Google, your site will appear in search results of AOL as well.

Almost all other search engines use the search index from one or the other above listed search engines to query and show results. So, as long as you are registered (and listed) by the major search engines listed above, you will also appear in search results of many other search engines.

So, don't just blog! Take another step and get other's to read and appreciate your blog by getting your blog searched from these commonly used search engines ;)

Monday, January 28, 2008

World famous fleet operator (package delivery) uses technology to cut costs and save Earth!

Have you ever taken a different route to your destination to escape crossings, right hand turns (in Indian context) or traffic?
Daily commuters in Metros these days would take to less-common routes (the road less traveled sometimes proves the road to success) to ensure an easy, congestion-free drive (saves time in most cases). But all such path discoveries happen either by exploring yourself or by a route-tip from friends(at least in India we're still limited in technology to have a live traffic beat). GPS and FM Radio Stations have started filling in but there's still a long way to go.
Just think of situations when we're waiting at the crossing (sometimes even to go straight/left which is supposed to be free), waiting for residual-traffic to clear off, engines all around revving up to cruise thru once the signal goes green... a waste of time and peace-of-mind (for all who're otherwise crunched of time in the work-place)... and also of fuel and money that goes in to fill that extra fuel.
But who cares, it's just a 2min wait, fuel wasted is negligible (I've a 100cc that runs 90kmpl).
Think about a lac big carrier trucks running in similar conditions all over the country-line that eat up not in liters but in gallons and that release a measurable amount of CO2 even when they're waiting to cruise! And what could be the cost-savings
when you own a gigantic fleet of vehicles delivering packages everyday?
Yes, small improvements in the efficiency of each one will translate to huge savings overall — and this is what led U.P.S. to limit further the number of left-hand turns (in US context) its drivers make.

Can't help but to highlight this article from NYT in my blog! One of the best use of technology towards the goal: "Let's make things better"

The company employs a “package flow” software, which among other hyper efficient practices involving the packing and sorting of its cargo, maps out routes for every one of its drivers, drastically reducing the number of left-hand turns they make (taking into consideration, of course, those instances where not to make the left-hand turn would result in a ridiculously circuitous route).

Last year, according to U.P.S., the software helped the company shave 28.5 million miles off its delivery routes, which has resulted in savings of roughly three million gallons of gas and has reduced CO2 emissions by 31,000 metric tons.

Monday, December 31, 2007

Some interesting blogs and sites i'll carry with me to 2008!

Come 2008 and the market/economic gurus predict a lot of volatility in everything related to money and forex. Not only this, technology will be at it's best to make our lives better (remember Philips saying: Let's make things better) and give us more options to communicate and collaborate on a wider scale. With all this, the concern and efforts for greener and safer earth will still be a "Work-In-Progress" and all this for the good of human.
Not looking at the flip side though(you can say I'm being too optimistic of a gr8 2008)

Well, all said, these are some of the web sites and blogs I see myself carrying with me to 2008 for a great compilation of topics covering a lot of areas that I go thru on a daily basis. Most of the links are just feeds so you can add them directly to your reader (I use Google reader).

A list of tech-blogs:
paulbridger.net - C++, patterns, design
Dr.Dobb's C++ Articles
DevX: Latest C++ Content
CodeGuru.com
Netotto Blog : Another Software Developer's Blog
Recording my programming path
Coding Misadventures
Monkey Bites

The ones on Network Security:
Dark Reading: Dark Reading News Analysis
SecGuru
SecurityFocus News

A list of blogs on Productivity, professional achievement and morale boosting things
Lifehacker: How To
Great Solutions to Team Challenges
Dumb Little Man - Tips for Life
Achieve IT!

A list of blogs on personal finance:
AllFinancialMatters
GetRichSlowly
I Will Teach You To Be Rich
Moneycontrol Pers Fin

Other than the above some weblinks that are interesting:
Koders
TBB
Shuva's Photo blog
Discover Magazine: Updates on Science
The Site for Books and Readers

With this and a lot more to come in 2008.... I wish all a very happy and green Year 2008!

Tuesday, December 18, 2007

How to use TBB parallel_for

Well, now that we've installed and configured TBB libs on our (Linux) machine, we can start playing with various parallelism constructs provided by TBB and use them to gain meaningful efficiency in our day-to-day problem solving.
Why stress on "meaningful"? We'll talk about it later in this column.

Here's a working code with execution timing difference b/w a multiplication job using sequential for and a TBB parallel_for. I've put in comments to explain use of important statements in the code.

//These includes are the ones reqd to use TBB lib's parallel_for
#include "/tbb/tbb20_20070927oss_src/include/tbb/blocked_range.h"
#include "/tbb/tbb20_20070927oss_src/include/tbb/parallel_for.h"
#include "/tbb/tbb20_20070927oss_src/include/tbb/task_scheduler_init.h"

#include < iostream>
#include < stdlib>
#include < sys/time.h>

#define MULTIPLIER 3.1456

//Just something I found at koders for timing
#define TIMERSUB(a, b, result) \
do { \
(result)->tv_sec = (a)->tv_sec - (b)->tv_sec; \
(result)->tv_usec = (a)->tv_usec - (b)->tv_usec; \
if ((result)->tv_usec < 0) { \
--(result)->tv_sec; \
(result)->tv_usec += 1000000; \
} \
} while (0)

using namespace tbb;
using namespace std;

typedef long long mytime_t;

// This structure contains an important part needed to be defined
// for TBB's parallel_for. We need to have UDT's with overloaded
// operators wrapping up the serial functionality that we want
// to break into parallel execution. In this case, I use a struct
// (could have used a class too) that encapsulates an array of
// input and output float numbers and has operator() defined to
// perform a serial for operation on input.


struct myNumbers {
float* ptr_input;
float* ptr_output;

//Just a constructor with an initialization list for use in the parallel_for call
myNumbers(float* input, float* output):ptr_input(input),ptr_output(output){}

// This is the actual body that's called in parallel_for by the TBB runtime.
// This code comes as a struct/class definition as the compiler
// expands and inlines this code as part of the template process.
// The TBB runtime takes the range blocked_range and breaks
// up the for loop in parallel threads to fit the the no. of
// processors/cores. The no. of processors/cores and thus the no.
// of threads to break this operation into are calculated by the
// TBB runtime and thus the developer using the TBB library can
// just concentrate on the functionality w/o worrying about the
// parallelizing math involved.

void operator()(const blocked_range &range)const {
for (int i = range.begin(); i!= range.end(); i++)
ptr_output[i] = ptr_input[i] * MULTIPLIER;
}
};

int main(size_t argc, char* argv[]) {
// for timing execution
timeval t_start, t_end, t_result, tbb_start, tbb_end, tbb_result;
mytime_t singlethread_time, tbb_time;
int i = 0;
float* ptr_input;
float* ptr_outputSingle;
float* ptr_outputTBB;

// Initialize the TBB runtime...
task_scheduler_init init;
if( argc != 2 ) {
cout<<"Usage: "<<< " \n";
return 1;
}
int numElements = atoi(argv[1]);
if( numElements <= 1 ) {
cout<<"Array size "<<<" reqd an integer > 1\n";
return 1;
}
ptr_input = new float[numElements];
ptr_outputSingle = new float[numElements];
ptr_outputTBB = new float[numElements];

for(i = 0; i < numElements; i++) {
ptr_input[i] = i;
ptr_outputSingle[i] = 0;
ptr_outputTBB[i] = 0;
}
//Time the execution using plain sequential for
gettimeofday(&t_start,NULL);
for( i=0; i < numElements; i++ ) {
ptr_outputSingle[i] = ptr_input[i] * MULTIPLIER;
}
gettimeofday(&t_end,NULL);
TIMERSUB(&t_start,&t_end,&t_result);
singlethread_time = (mytime_t)(t_result.tv_sec + t_result.tv_usec);

//Time the execution using TBB parallel_for
gettimeofday(&tbb_start,NULL);
parallel_for(blocked_range(0,numElements),
myNumbers(ptr_input,ptr_outputTBB), auto_partitioner());
gettimeofday(&tbb_end,NULL);
TIMERSUB(&tbb_start,&tbb_end,&tbb_result);
tbb_time = (mytime_t)(tbb_result.tv_sec + tbb_result.tv_usec);

//Verify that the outputs match
for(i=0; i < numElements; i++) {
if( ptr_outputSingle[i] != ptr_outputTBB[i] ) {
cout << ptr_input[i] << " * " << MULTIPLIER <<" = " <<
ptr_outputSingle[i] << " AND " << ptr_outputTBB[i] << endl;
}
}

cout << "Sequential for execution time: " << singlethread_time << " units"<< endl;
cout << "TBB parallel_for execution time: " << tbb_time << " units" << endl;
return 0;
}

I used a Makefile for this, but we can also do it as (assuming the file as simple_for.cpp):
g++ -O2 -DNDEBUG -o ./simple_for simple_for.cpp -ltbb

Now, let's talk about "meaningful" efficiency gains:
I ran this code on my Linux machine with args (numElements) as: 10, 100, 1000 and found some performance improvement when using TBB parallel_for (assuming that execution timings are reported correctly). But when I ran this for numbers beyond 10000, I found that serial for does it in lesser time. No No No buddy.... don't even think about parallel_for having a constraint here in terms of breaking things up into smaller chunks of serial for(s). Intel (and others who support parallelism) specifically take into account Amdahl's Law and Gustafson's Law while proposing TBB to developers. So there's some level of optimization provided in TBB (based on practical load/figures and processor configuration that one's using). Like in this case I could overcome this limit of 10000 by providing a "grainsize" to the blocked_range() constructor as:
parallel_for(blocked_range(0,numElements,10),myNumbers(ptr_input,ptr_outputTBB));
Here, the 3rd arg to blocked_range() is grainsize and I saw more performance improvements for larger iterations if I keep reducing it (finally to 10) starting from an initial grainsize of 1000. Also observe that I've not used auto_partitioner arg in the call to parallel_for if I use grainsize in blocked_range constructor. Using a partitioner for deciding range of parallel chunks for your processing subsystem is one of the new features provided in TBB and with auto_partitioner the TBB runtime chooses a chunk size automatically optimized for parallelizing iterations on the underlying processing subsystem.

Refer TBB Getting Started Doc for more details on how to select the right grainsize for your iterations and for partitioner details.
In short: grainsize specifies the number of iterations for a “reasonable size”
chunk to feed the processor. If the iteration space has more than grainsize iterations, parallel_for splits it into separate subranges that are scheduled separately.

Yo! So we'd a gr8 start with TBB parallel_for demonstrating it's might in multicore/multi-processor environments. There's a lot more to parallel algos in TBB library. Not only that, there's a whole bunch of customized STL constructs that now work in tandem with multi-threaded code w/o the developer worrying about maintaining the threading infrastructure. Lemme explore some more of these features next week when I come back after the X-mas vacations. Till then, happy parallelizing!!!

Thursday, December 13, 2007

Fun with Intel TBB!

Phew! With Linux there's always some amount of configuration/tweaking reqd. before you can build source or make use of a new library...And I must tell you, I luv this challenge! The best example of this is when you want a GNU app or a framework to help you do something more with your Linux box. Most of the time we'll download src code from GNU free s/w websites like sourceforge etc to get started. And then starts the process to configure, make the src and install it. That's not all, sometimes you've to go a step further and comment or fix some simple errors (like casting) in the C files of the app before you can successfully build the app and get the reqd. binaries.
Here I want to capture an experience I had with installing TBB on my Linux box (running SLES 10 with 2.6.16.21 kernel and gcc/++ version 4.1.0)
Let's go step-by-step from here:
1) copy the following tar.gz files to some folder like /tbb/
tbb20_20070927oss_src.tar.gz
tbb20_20070927oss_lin.tar.gz
2) Now, extract everything there itself using tar -zxvf filenames
3) This will give you two folders:
tbb20_20070927oss_src
tbb20_20070927oss_lin
4) From tbb20_20070927oss_lin copy the folder ia32 to tbb20_20070927oss_src directory (it's a 32 bit platform on an intel box)
5) If you're lucky enuf you'll get the libtbb.so and others for your kernel+glibc version in one of the four folders inside: tbb20_20070927oss_src/ia32
6) If not, we need to build the libtbb.so (the crux of everything) for your platform, so "cd /tbb/tbb20_20070927oss_src/src/tbb/"
7) Run "make" here and see if your luck strikes, to get a libtbb.so w/o errors.
8) If not, then try either of these things or both:
(a) If you see a make Error for task.cpp then you may be asked to fix this:
/src/tbb/task.cpp:396: warning: deprecated conversion from string constant
I know you can do this, so I won't fix it for you here ;)
(b) If it still doesn't work then figure out what else is preventing a successful make of libtbb.so and try resolve it.
Lastly, you can try using the libtbb.so from any of the ia32 folders like: tbb/tbb20_20070927oss_src/ia32/cc4.1.0_libc2.4_kernel2.6.16.21/lib
9) Once you've the right version of libtbb.so and libtbbmalloc.so for your platform, create their soft links in /usr/lib/
10) Now, we're ready to make a sample code supplied with TBB src.
Goto sample code folder "cd tbb/tbb20_20070927oss_src/examples/parallel_for/seismic" and do a make here.
11) Again, things are not that straight buddy!
You need to either add to Makefile the include path for files included in Seismic.cpp like /tbb/tbb20_20070927oss_src/include/tbb/parallel_for.h or edit the .cpp file to have absolute path to these .h files
12) After fixing all these make dependencies, you'll be able to build the binary and see it running on your Linux machine with figures telling you no. of fps with parallelism.

Now that we've the machine run this example successfully, why not try our own parallel_for which seems to be a good starting point to go parallel the Intel way!!
Coming up next -> How to use TBB parallel_for

Tuesday, October 30, 2007

Network Security: Beware! (Part 1)

This one is the first in a series that highlights various aspects of n/w security that goes un-attended or needs still more attention!

I was going thru some articles on N/w security as part of my daily Security-dose. As a developer working on products that uses the word "Security" day-in-day-out... I feel that we sometimes overlook the importance of this word in our security products' implementation.
There's this book Stealing the Network: How to Own the Box that reveals (in a fictional story telling way, and i like it) different ways of gaining unauthorized access to secure Networks and computer systems. Another read about writing secure code, adresses why programmers write in-secure code. It also highlights common and well understood exploits/issues of the past (due to C being unsafe in certain areas) that still go un-checked in every 2nd release.

Not all the ownership lies with the developer looking at the fact that Strategic enhancements/design/architecture is still a work of a few evangelists in the field, but as far as the hacks caused by bugs in the code are concerned they go much in number; big names in the security products industry seem have a direct proportion b/w the brand value and the no. of exploits in their product.

Avenues of attack:
As far as avenues of attack within the secure n/w are concerned Printers now-a-days are not behind. Like our ever-increasing-powerful PC, printers (read multi-function devices with copy-fax-print-send_fax_as_email) too are becoming intelligent agents in the enterprise. What if they started out as passive devices like the old character/line/chain printers, printers have now evolved into actively managed network agents that run pretty neat OS kernels which supports the network stack software and very well understand the Enterprise n/w hierarchy allowing identity-managed print jobs, authorised administration using print-servers etc.
There is still a tendency to take them as the passive utility devices that explains the negligible network security on a printer. Since the printer can communicate with the rest of the network, it can serve as a platform for attack (as a network proxy if nothing else).

As Bruce Schneier mentions in his foreword to Building Secure Software: How to Avoid Security Problems the Right Way:
"We wouldn't have to spend so much time, money, and effort on network security if we didn't have such bad software security. Think about the most recent security vulnerability about which you've read. Maybe it's a killer packet that allows an attacker to crash some server by sending it a particular packet. Maybe it's one of the gazillions of buffer overflows that allow an attacker to take control of a computer by sending it a particular malformed message. Maybe it's an encryption vulnerability that allows an attacker to read an encrypted message or to fool an authentication system. These are all software issues."

At the 2007 RSA Conference, Bruce puts another point: Human beings aren't evolved for security in the modern world, and particularly the IT security world. There is a gap between the reality of security and the emotional feel of security due to the way our brains have evolved. This leads to people making bad choices.

Towards this state of N/w security, a lot can be attributed to software bugs; the way we've been writing OS, N/w applications and middle-tier code.

The language used to develop most of the OS's (not only Win, UNIX/Linux but also OS's that run on the routers, printers, ATM's etc.) had been C/C++ that had limitations in terms of memory leaks (GC features),compiled range checks etc. (although we've a lot of evolved features now in the language). For e.g.: Buffer overflow attack results (mostly) due the missing compiled range checks in C/C++
With range check capability a buffer (array) overflow can be caught at runtime.
Java compiler inserts code that checks access to a buffer (array) is within the bounds of the allocated memory. If the access goes past the end of the buffer a runtime error should occur. It's better to error/exit/raise an exception in the software rather than let the intruder gain access to the system.

char mySockBuffer[SOME_DEFINED_SIZE];
while (!read_my_socket(mySockBuffer)) {
do_something();
}
The code reads a stream of text from an input connected to a TCP/IP socket. A stream of text longer than 80 characters will overwrite data which is stored following mySockBuffer. A text stream of just the right size may overwrite the return address for a function (possibly the function that called the current function), allowing the attacker to insert and execute his own code.
This can result in a potential BOEP and can even let an attacker write over the legitimate instructions to get a malicious code executed on the machine.
A typical example is a code where the developer has a 256 characters array to hold a login username for a web based back-end system.
A hacker sends 300 characters with code that will be executed by the server, and voila, he has broken in. Hackers can find these bugs in many ways.
1) the source code for a lot of services (GNU based are popular) is available on the net. Hackers routinely look through this code searching for programs that have buffer overflow problems.
2) hackers may look at the programs themselves to see if such a problem exists, though reading assembly output is really difficult.
3) hackers will examine every place the program has input and try to overflow it with random data. If the program crashes, there is a good chance that carefully constructed input will allow the hacker to break in.

An intruder would try using unexpected combinations of input on web based forms to break in. Most of the apps today use multiple layers of code, that includes API calls to the underlying operating system as the bottom most layer. An intruder can send the input that is interpreted as a string by one layer, but taken as a meaningful command by another. For e.g. PERL is mostly used for processing user inputs on web based systems. PERL usually sends this input to other programs for further evaluation. A common hacking technique would be to enter something like "| mail < /etc/passwd". This gets executed because PERL asks the operating system to launch an additional program with that input. However, the operating system intercepts the pipe '|' character and launches the 'mail' program as well, which causes the password file to be emailed to the intruder (although not so easy as it looks). I've seen instances where intruders have tried breaking in by this route where they can execute their malicious code as a legitimate piece of response to an event on a system.
Exceptions and Unhandled input: Most programs are written to handle valid input with known preconditions and evaluations. A developer won't always consider what happens when someone enters input that doesn't match the specification. This could become a security hole that an intruder can exploit for an unexpected program behavior during unhandled input scenarios.

There are certain other scenarios and known exploits for Network security apps, OS and n/w configuration. There are design flaws in N/W protocols and loop-holes in the way OS security system works. Just think about TCP/IP... it was designed earlier when there was not much of a wide-spread intrusion/hacking concern thus leaving enuf ways for intruders like: smurf attacks, ICMP Unreachable disconnects, IP spoofing, and SYN floods. The biggest problem is that the IP protocol itself is very "trusting": anyone can forge and change IP data with impunity. IPsec (IP security) has been designed to overcome many of these flaws, but it is not yet widely used.

What next: I'll be compiling some programming tips, DO's and DONT's etc. on n/w security and putting them all in the next post on this series. Bugs too will make it to the next one.
For more similar reads refer:
http://www.linuxsecurity.com/resource_files/intrusion_detection/network-intrusion-detection.html
http://www.embedded.com/design/202300629?pgno=1

Tuesday, October 23, 2007

Useful RPM options

rpm, search for this in Wikipedia and you'll get some gr8 info about revolutions per minute. But as we all know this tool and have been using this for quiet some time now on Linux distros...
Here's what www.rpm.org has to say about RPM:
The RPM Package Manager (RPM) is a powerful command line driven package management system capable of installing, uninstalling, verifying, querying, and updating computer software packages. Each software package consists of an archive of files along with information about the package like its version, a description, and the like. There is also a library API, permitting advanced developers to manage such transactions from programming languages such as C or Python.

With that bit of knowledge about RPM let me assure you that even a non-programmer would be doing a lot on his Linux machine that needs him to know basic rpm usage.
Like: 1) rpm -ivh for installing a new RPM pkg
2) rpm -Uvh for upgrading an existing rpm pkg
3) rpm -e for un-installing an rpm
4) rpm -Uvh --test To test a package to see how it would install (w/o installing, also checks dependencies)

Now, there's a hell lot of information on how to do things with RPM at:
Maximum RPM
Just so that I can remember some useful but difficult-to-find-when-you-need'em options, I've tried to put up a collection here:

RPM Querying: Get Info from installed Packages

To see a list of all installed packages: rpm -qa | less

Don't know what a specific installed package does? Hey, tell me about yourself:
rpm -qi
To know what files were installed by a specific installed package: rpm -ql
A similar thing is also achievable by: rpm -q --filesbypkg
To know the config files in an installed package: rpm -qc
There's a file on my comp /usr/lib/foo-lib. To find out which installed package it belongs to: rpm -qf /usr/lib/foo-lib
To find out which package installed the above file AND to get information on that package and see all the other files it installed: rpm -qilf /usr/lib/foo-lib
The scripts in a package: rpm -q --scripts
Services that this package provides: rpm -q --provides
Services that this package requires: rpm -q --requires

For pkg that has NOT been installed yet, we can query for similar information by adding the -p option to the commands listed above.
Get information about an RPM pkg (not yet installed) and the files it would install: rpm -qilp

Verify options:
To verify a package (with lots of verbose output): rpm -Vvv
To verify the cryptographic signature of a yet-to-be-installed package:
rpm -K
And to test the integrity of a yet-to-be-installed pkg: rpm -K --nopgp package.rpm
To verify ALL installed packages on the comp: rpm -Va
Imp Tip: The above command is also useful in the following scenarios:
1) You deleted some files accidently but you don't know what they are. The above rpm command can show you the files that are now missing in its database.
2) Think you've been hacked? To check for files that have been modified or removed in any way for any installed RPM packages.

To extract an individual file from an rpm package without installing the rpm:
1. Use rpm2cpio or rpm -qpl to list files and full paths in the package:
rpm2cpio package | cpio -t
Now, use the full path name of a file listed above to extract it in step 2.
2. Use rpm2cpio to extract a file. Run this command from your home directory or /tmp in order to avoid overwriting any current system files.
rpm2cpio package | cpio -iv --make-directories
This creates the full path in the current directory and extracts the file you specified.
3. If you just want to convert it to a cpio archive, use
rpm2cpio package > cpio-archive-file

To extract all the files from an RPM package:
rpm2cpio package | cpio -i --make-directories

Although there are a lot of other useful options, I'll have these most useful ones for now on this page.
Some imp links for rpm options:
http://dave.thehorners.com/content/view/111/65/
http://susefaq.sourceforge.net/articles/rpm.html

I would be interested to know some scenarios where we use a complex set of options to determine more practical information on rpm installs.
Also one good link (place holder for me) about un-installing a gnu-source compiled program:
http://blog.netotto.com/index.php?entry=entry071020-232245

Tuesday, October 16, 2007

Tracking down those hidden startup processes in Windows


Is your Windows being dragged by un-necessary startup apps eating up resources? Are you compromising your privacy by not knowing what spywares are tracking your actions on the web? Need an effective tool for tracking those DLLs, services and applications that automatically load at system startup?

Here's the tool:

Sysinternals’ Autoruns, a free troubleshooting application being provided now by Microsoft!

This tool offers an easy to use yet powerful GUI that tracks spywares, processes... moreover the DLLs, services, applications and other critical stuff that load behind the scene on Windows systems.
With this tool, keeping a check on your startup apps, removing unwanted dll-loads/services/free-wares/ trial-expired s/w or adding service entries is a lawn-walk. It offers check-boxes to enable/disable each and everything, with a description of the Publisher of that s/w, service, dll and the location of the binary/library being tracked.


Some evaluated Pros & Cons from Techrepublic:

+ve
* price (it’s free!)
* Simple installation
* Administration is easy, thanks to a straightforward GUI
* Thorough tracking of installed and active processes

Less than helpful in scenarios like:
* Some malware applications may not register within active processes, rendering Autoruns less than helpful when combating particularly problematic infections
* Deleting processes won’t remove all remnants of many unwanted programs from the hard disk
* Removing infections that infest multiple user accounts may need to be removed as many times as there are user accounts

A gr8 tool?

With this gr8 tool for tracking down things running on your machine behind your back...being offered free by MS... It becomes a good one in the list of have's for a sys-admin or a helpdesk engineer (also for a normal Windows freak).
With a thorough coverage and dependable performance, this free utility is the right tool for almost any malware troubleshooting routine (also could be used to easily tweak system performance)

Download and assess yourself: http://download.sysinternals.com/Files/Autoruns.zip

Usage. Point of views?

Friday, September 28, 2007

Purpose

This is my first tech blog. I'll post tips/techniques/articles/thoughts on varied aspects of technology that I use and learn on a day-to-day basis.

I won't have a disclaimer or a copyright for any of my collections/posts on this blog... one being a share-n-grow-knowledge believer and other being a gnu/free s/w and services user and promoter.