Update on Problems installing VSTS 2005 Dec. 2004 Refresh

If you recall my previous post on VSTS installation blues, I got an answer from MS on that. It seems like installing Yukon Beta 2 did it. Per Microsoft, uninstalling Yukon won’t also help and it is better to rebuild the OS and then go from there. You can read it on the discussion thread here. I guess, there goes another evening in rebuilding a machine.

Simplified Data Processing on Large Clusters

Jeffrey Dean and Sanjay Ghemawat had written a very interesting paper highlighting their research in Google for Simplified Data Processing on Large Clusters. Their implementation called MapReduce is a programming model and an associated implementation for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key.

Programs written in this functional style are automatically parallelised and executed on a large cluster of commodity machines. The run-time system takes care of the details of partitioning the input data, scheduling the program’s execution across a set of machines, handling machine failures, and managing the required inter-machine communication. This allows programmers without any experience with parallel and distributed systems to easily utilise the resources of a large distributed system.

The programming model they cover takes a set of input key/value pairs and produces a set of output key/value pairs – all of these intermediate values are clubbed together along with a key which in itself could possibly be a smaller set of values. On the implementation front, as they rightly point out, the right choice depends on the environment (one implementation might work for a large cluster of machines, another thought might be better for a small shared memory machine). Execution Overview shows the overall flow of a MapReduce operation. When the user program calls the MapReduce function, the following sequence of actions occurs as shown in the paper.

  1. The MapReduce library in the user program first splits the input files into M pieces of typically 16 megabytes to 64 megabytes (MB) per piece (controllable by the user via an optional parameter). It then starts up many copies of the program on a cluster of machines.

  2. One of the copies of the program is special – the master. The rest are workers that are assigned work by the master. There are M map tasks and R reduce tasks to assign. The master picks idle workers and assigns each one a map task or a reduce task.

  3. A worker who is assigned a map task reads the contents of the corresponding input split. It parses key/value pairs out of the input data and passes each pair to the user-defined Map function. The intermediate key/value pairs produced by the Map function are buffered in memory.

  4. Periodically, the buffered pairs are written to local disk, partitioned into R regions by the partitioning function. The locations of these buffered pairs on the local disk are passed back to the master, who is responsible for forwarding these locations to the reduce workers.

  5. When a reduce worker is notified by the master about these locations, it uses remote procedure calls to read the buffered data from the local disks of the map workers. When a reduce worker has read all intermediate data, it sorts it by the intermediate keys so that all occurrences of the same key are grouped together. The sorting is needed because typically many different keys map to the same reduce task. If the amount of intermediate data is too large to fit in memory, an external sort is used.

  6. The reduce worker iterates over the sorted intermediate data and for each unique intermediate key encountered, it passes the key and the corresponding set of intermediate values to the user’s Reduce function. The output of the Reduce function is appended to a final output file for this reduce partition.

  7. When all map tasks and reduce tasks have been completed, the master wakes up the user program. At this point, the MapReduce call in the user program returns back to the user code.

To further quote Jeffrey and Sanjay, after successful completion, the output of the mapreduce execution is available in the R output files (one per reduce task, with file names as specified by the user). Typically, users do not need to combine these R output files into one file . they often pass these files as input to another MapReduce call, or use them from another distributed application that is able to deal with input that is partitioned into multiple files.

Over the past five years, the authors and many others at Google have implemented hundreds of special-purpose computations that process large amounts of raw data, such as crawled documents, web request logs, etc., to compute various kinds of derived data, such as inverted indices, various representations of the graph structure of web documents, summaries of the number of pages crawled per host, the set of most frequent queries in a given day, etc. Most such computations are conceptually straightforward. However, the input data is usually large and the computations have to be distributed across hundreds or thousands of machines in order to finish in a reasonable amount of time. The issues of how to parallelise the computation, distribute the data, and handle failures conspire to obscure the original simple computation with large amounts of complex code to deal with these issues.

As a reaction to this complexity, we designed a new abstraction that allows us to express the simple computations we were trying to perform but hides the messy details of parallelisation, fault-tolerance, data distribution and load balancing in a library. Our abstraction is inspired by the map and reduce primitives present in Lisp and many other functional languages. We realised that most of our computations involved applying a map operation to each logical ?record? in our input in order to compute a set of intermediate key/value pairs, and then applying a reduce operation to all the values that shared the same key, in order to combine the derived data appropriately. Our use of a functional model with userspecified map and reduce operations allows us to parallelise large computations easily and to use re-execution as the primary mechanism for fault tolerance. The major contributions of this work are a simple and powerful interface that enables automatic parallelisation and distribution of large-scale computations, combined with an implementation of this interface that achieves high performance on large clusters of commodity PCs.

Their implementation of MapReduce runs on a large cluster of commodity machines and is highly scalable: a typical MapReduce computation processes many terabytes of data on thousands of machines. Programmers find the system easy to use: hundreds of MapReduce programs have been implemented and upwards of one thousand MapReduce jobs are executed on Google’s clusters every day.

To get more details, check out their paper.

[Listening to: Tabba (Techno) – Musafir [Club Mix – CD1] – Musafir (05:49)]

Is your son a hacker?

I forgot how I got this, it was thru someone but don’t recall. Whoever told me this, please ping me if you are reading this so I can give you credit. Anyways Gibbons has a very enlightening read which every parent who has a computer at home, just have to read! Here is an excerpt:

As an enlightened, modern parent, I try to be as involved as possible in the lives of my six children. I encourage them to join team sports. I attend their teen parties with them to ensure no drinking or alcohol is on the premises. I keep a fatherly eye on the CDs they listen to and the shows they watch, the company they keep and the books they read. You could say I’m a model parent. My children have never failed to make me proud, and I can say without the slightest embellishment that I have the finest family in the USA.

Two years ago, my wife Carol and I decided that our children’s education would not be complete without some grounding in modern computers. To this end, we bought our children a brand new Compaq to learn with. The kids had a lot of fun using the handful of application programs we’d bought, such as Adobe’s Photoshop and Microsoft’s Word, and my wife and I were pleased that our gift was received so well. Our son Peter was most entranced by the device, and became quite a pro at surfing the net. When Peter began to spend whole days on the machine, I became concerned, but Carol advised me to calm down, and that it was only a passing phase. I was content to bow to her experience as a mother, until our youngest daughter, Cindy, charged into the living room one night to blurt out: “Peter is a computer hacker!”

[Listening to: ChhodNaRe(Rimix) – – (05:01)]

Microsoft Antispyware (Beta 1) screenshots

I installed Microsoft’s Antivirus (Beta 1) over the weekend and here are some screen shots from the installation. Btw, I did not find anything on my machine, and honestly on this machine, I did not have any intentions of trying that out.

Coolest memory ever – Modder's Rejoice

Extremetech is reporting about Corsair Memory’s new XPERT Memory has one seriously distinguishing feature: It has a 10 character alphabetical LED display that snaps in place over your DIMMS. Install the RAM in your system and it automatically offers up your your memory speed, voltage, and DIMM temperature in bright red characters. Remember calculators in the Seventies? It’s that kind of LED display, and it really stands out through a case window.

Better yet, you can use Corsair’s Memory Dashboard to program the memory to display your own personalized greeting (or obnoxious salute). Type in the message, click a button, and your message gets sent to some spare bits on the memory. And from then on, that message will scroll across the display like your very own tiny Times Square Zipper…except it won’t be giving you the latest news headlines and sports scores.

Problems installing VSTS 2005 Dec. 2004 Refresh

I am sure you also have played with VSTS, but not sure if you are having problems with VSTS Dec 2004 Refresh bits up on MSDN. I am installing this natively (not on VMWare/VirtualPC, etc.) running Windows 2003 Enterprise Edition with VS.NET 2003, SQL Server 2000, SQL Server 2005 (Beta 2), and usual suspects like Office 2003, etc. During the installation process when I get to installing Visual Studio (see screen shots below), aftre going about 60-70% of the way I get an error – “Product: Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English — Error 1304.Error writing to file: Microsoft.VisualStudio.Windows.Forms.dll.  Verify that you have access to that directory.” Now everything seems to be ok to me, I am logged as God (Domain Admin) on the box and there is enough disk space as well, so not sure what is causing this.

Here are the screen shots (you can click on them to see the full resoultion):

When I look at the logs here is the error from there. You can download the complete installation log from here.

[01/07/05,08:41:31] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: CRootComponent::SetMsiLogFileName(): Setting MSI Log Name to:C:\Documents and Settings\Administrator\Local Settings\Temp\VSMsiLog2838.txt
[01/07/05,08:41:31] Setup.exe: AddGlobalCustomProperty
[01/07/05,08:59:13] vs70uimgr: Entering DisplayMessage() method.
[01/07/05,08:59:14] vs70uimgr: DisplayMessage_START:Error 1304.Error writing to file: Microsoft.VisualStudio.Windows.Forms.dll.  Verify that you have access to that directory.
[01/07/05,09:04:00] vs70uimgr: DisplayMessage_END:Error 1304.Error writing to file: Microsoft.VisualStudio.Windows.Forms.dll.  Verify that you have access to that directory.
[01/07/05,09:04:00] Setup.exe: AddGlobalCustomProperty
[01/07/05,09:04:00] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: ***ERRORLOG EVENT*** : ERROR:Error 1304.Error writing to file: Microsoft.VisualStudio.Windows.Forms.dll.  Verify that you have access to that directory.
[01/07/05,09:04:00] vs70uimgr: Entering DisplayMessage() method.
[01/07/05,09:04:00] vs70uimgr: DisplayMessage_START:The installer has encountered an unexpected error installing this package. This may indicate a problem with this package. The error code is 2350.
[01/07/05,09:04:01] vs70uimgr: DisplayMessage_END:The installer has encountered an unexpected error installing this package. This may indicate a problem with this package. The error code is 2350.
[01/07/05,09:04:01] Setup.exe: AddGlobalCustomProperty
[01/07/05,09:04:01] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: ***ERRORLOG EVENT*** : ERROR:The installer has encountered an unexpected error installing this package. This may indicate a problem with this package. The error code is 2350.
[01/07/05,09:04:02] Setup.exe: AddGlobalCustomProperty
[01/07/05,09:04:02] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: ***ERRORLOG EVENT*** : ACTION FAILURE:Action ended 09:04:02: InstallFinalize. Return value 3.See MSI log for details.
[01/07/05,09:09:26] Setup.exe: AddGlobalCustomProperty
[01/07/05,09:09:26] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: ***ERRORLOG EVENT*** : ACTION FAILURE:Action ended 09:09:26: INSTALL. Return value 3.See MSI log for details.
[01/07/05,09:09:36] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: CRootComponent::Install(): Finished MSI Installation
[01/07/05,09:09:36] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: MsiInstallProduct return code: 1603.
[01/07/05,09:09:36] InstallReturnValue: root_comp, 0x643
[01/07/05,09:09:36] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: CRootComponent::Install(): Setup Failed; MSIInstallProduct return value either ERROR_INSTALL_FAILURE or default.
[01/07/05,09:09:36] Setup.exe: AddGlobalCustomProperty
[01/07/05,09:09:36] Setup.exe: AddGlobalCustomProperty
[01/07/05,09:09:36] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: CRootComponent::Install(): Calling LaunchWatson()…
[01/07/05,09:09:37] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: CRootComponent::LaunchWatson() – Obtained CSetupWatson instance
[01/07/05,09:09:37] Setup.exe: GetGlobalCustomProperty({B8A13F14-4BDE-4EC4-A53C-8E51FBB8261F})
[01/07/05,09:09:37] Setup.exe: GetGlobalCustomProperty({4F8791EE-486F-44B5-952B-F28F9FD58BD0})
[01/07/05,09:09:37] Setup.exe: GetGlobalCustomProperty({710A502B-4669-405C-A985-BA21545509BA})
[01/07/05,09:09:37] Setup.exe: GetGlobalCustomProperty({710A502B-4669-405C-A985-BA21545509BA})
[01/07/05,09:09:37] Setup.exe: GetGlobalCustomProperty({AA62DF98-3F2C-11D3-887B-00C04F8ECDD6})
[01/07/05,09:09:37] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: CRootComponent::LaunchWatson() – Launching VS Watson
[01/07/05,09:10:34] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: CRootComponent::LaunchWatson() – Completed VS Watson launching to create manifest: C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\vs_setup.dll.txt
[01/07/05,09:10:34] Setup.exe: GetGlobalCustomProperty({D530994B-CA3E-4CF3-AC2F-485962898182})
[01/07/05,09:10:34] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: CRootComponent::LaunchWatson() – Cleaning up the VS Manifest; some other Manifest is ready to be used.
[01/07/05,09:10:39] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: CRootComponent::Install(): Finished calling LaunchWatson()
[01/07/05,09:10:39] Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English: CRootComponent::Install(): Finished Brooklyn Component Installation
[01/07/05,09:10:39] Setup.exe: AddGlobalCustomProperty
[01/07/05,09:10:39] setup.exe: ***ERRORLOG EVENT*** : ISetupComponent::Pre/Post/Install() failed in ISetupManager::InstallManagerHelper() with HRESULT -2147023293.
[01/07/05,09:10:39] Setup.exe: AddGlobalCustomProperty[01/07/05,09:10:39] setup.exe: ***ERRORLOG EVENT*** : Component error string not specified in ISetupManager::AddToActionResultCollection()
 

If you look at the event log (you can see the entries being written above) and essentially it shows the same information. Here are the two entries there:

First Entry:

Event Type: Error
Event Source: MsiInstaller
Event Category: None
Event ID: 11304
Date: 09/01/2005
Time: 12:08:26 AM
User: N/A
Computer: AMIT-D2E6YEDPFS
Description:
Product: Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English — Error 1304.Error writing to file: Microsoft.VisualStudio.Windows.Forms.dll. Verify that you have access to that directory.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 7b 43 46 33 32 33 34 44 {CF3234D
0008: 30 2d 38 36 35 30 2d 34 0-8650-4
0010: 45 33 37 2d 41 34 42 45 E37-A4BE
0018: 2d 33 44 35 38 32 32 36 -3D58226
0020: 39 42 45 41 34 7d 9BEA4}

Second Entry:

Event Type: Error
Event Source: MsiInstaller
Event Category: None
Event ID: 10005
Date: 09/01/2005
Time: 12:08:28 AM
User: N/A
Computer: AMIT-D2E6YEDPFS
Description:
Product: Microsoft Visual Studio 2005 Team Suite Edition Beta 2-English — The installer has encountered an unexpected error installing this package. This may indicate a problem with this package. The error code is 2350. The arguments are: , ,

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 7b 43 46 33 32 33 34 44 {CF3234D
0008: 30 2d 38 36 35 30 2d 34 0-8650-4
0010: 45 33 37 2d 41 34 42 45 E37-A4BE
0018: 2d 33 44 35 38 32 32 36 -3D58226
0020: 39 42 45 41 34 7d 9BEA4}

Microsoft has a KB that talks about troubleshooting Visual Studio.NET installation, but that really not helpful in this context.

So, has anyone else seen it? Having talked to some of my buddies, it seems like they did not have an issue. I thought the extract of the ISO could be corrupt, so I extracted it again (though I did not download the 3.15 gb again) and I cannot think what the issue might be.

Tips and Trips for .NET Compact Framework

Dave and John discuss various aspects of the .NET Compact Framework (CF) where in addition to the list belowthey cover things like the fastest way to splat text onto the screen? ExtTextOut using which you can draw text based on the currently selected font, background color, and text color. You can optionally provide dimensions to be used for clipping, opaquing, or both. It can also draw with a transparent background, which labels in the .NET Compact Framework do not support (just try to put a label over a PictureBox and you’ll see).

  • Tips for making programming in the .NET CF environment easier
  • Tricks for making your Pocket PC-based apps run faster
  • Important .NET CF classes
  • .NET CF windowing issues

If you do development on the CF then this is good article to read up on. As they conclude “Writing applications for the .NET Compact Framework is a wonderful experience. The applications are small enough that shareware developers can once again write something on their own and sell it online to make a little bit of money. But the simplicity and small capacity of mobile devices present new challenges to programmers. Without awesome power, you have awesome responsibility. Strive from the get-go to design code that is fast, tight, and highly, highly disposable.

More Information:  http://tinyurl.com/6xvsl

Ads on MSDN?

This is not a big deal, but does have me ticked, why does Microsoft have to sell ads on MSDN?? Are they running low on the cash on the coffers? Honestly this was the last place I expected it and is quite irritating. Check out the screen shot with the two arrows pointing to the ads. (You can click on the image to see in full-size).

Vonage launched in UK

Sweet, now that Vonage has launched in the UK, I am thinking of getting it. My US one worked out pretty good for me and have been using it more than a year and a half. But, since I am still very new in the UK, for the old timers here, what do you think of their rates? For £9.99 per month I can make unlimited local and national calls anywhere in the UK and Ireland and the rates for mobile are 15p (day), 10p (evening) and 5p (weekend). Plus the international calling rates are pretty cheap (or so I think). Thoughts?

[Listening to: Hook – Blues Traveler – (3:38:32)]

Firefox Extensions

I have my own favorite extensions, but Luke discusses more on his post here. Interestingly with Firefox being safer and all that (compared to IE) Spoofstick is missing from most lists.

[Listening to: Chapter 2 – Unknown – (01:52:32)]