This has been a subject I have thought about for a very long time. Actually it is much more than thinking. It is also doing. In the past fifteen years I suspect I have written at least 1/2 million lines of code across a number of different languages. Everything from simple toycode to gnarley programs that do cross tabulation, or programs that convert and flatten proprietary hierarchical dictionary data [convert and flatten proprietary hierarchical dictionary]
There is a definite element of art to programming. If you do not wish to call it art, then one needs to accept that it as at least craftsmanship. Writing a quality program, or abstracting and developing a killer library takes a great deal of intellectual capital. There is enormous amounts of creativity involved in looking at and solving problems in code. For me not to consider this discipline as art is up there with stating that accounting and taxes are nothing more than problems we solve with math.
Ha! Have you ever known math that mostly boils down to simple arithmetic to be so mind bending?
Our buds at Wikipedia say that "Art is the product or process of deliberately arranging symbolic elements in a way that influences and affects the senses, emotions, and/or intellect."
They go on to describe traditional things like music, painting, photography as art. I own a bunch of different types of cameras and dabble as a rank amateur in photography. I see many parallels between the two disciplines. The process involved in setting up a shot. Is it a worthy subject? Is the light right? How do you set everything up so the film records the same thing, the same experience you are witnessing? So many elements of design to consider.
If you are developing an application, and you are working with more than a narrow set of design inputs you are dealing with a similar canvas. There are numerous ways in which to can go about solving the problem. The technologies (the medium) you have to work with dictate how you will work. As you explore the possibilities and endless alternatives you begin to see patterns emerge. Will you, or how will you deconstruct and reuse these patterns? Again, so many elements of design to consider.
So we don't have galleries for our work. So much of the thought and design is only seen and appreciated by our peers and contemporaries. So what?
Art does not always have to be overt. Art does not always need to be for the masses. Art does not always have to be 100% pure. We call music art. Yet there are styles and genres (Jazz, Reggae, Rock, Country, Hip-Hop) that are nothing more than patterns we arrange. These higher order patterns are based upon a finite set of chords and notes and tones and symbols that we arrange and rearrange. Is programming that much different?
Okay, our net result is not something we can construe as art. But the process we go through to design and create can be. Meaning, it not always is.
I know many programmers that would not consider what they do to be art. And, I would concur that most coding is not art, and most people that code are not artists. This is reasonable. Not everyone that owns a camera and takes pictures is an artist either.
In the end I am not sure it really matters whether what we do is considered art or not. What matters most is that it is something we enjoy doing.
Tuesday, October 26, 2010
Wednesday, August 25, 2010
How Hardware Storage Increases Have Changed Software
Three weeks ago I bought a 750gb external drive for around $100, including tax. Three days ago I saw an even better deal for the same size. The cheap cost of space is really quite astounding if you look back in time.
5 years ago... Well, my current computer, I am about to swap out this friday has 80gb
15 years ago... My work issued computer was a 486dx-32mhz with a 120mb drive. Whohooo, or as they say in the Bronx, "fuuugettaboutit"
I remember back in the late nineties building web content and worrying about what impact adding a 3k or 4k graphic would do. Today I am having a hard time keeping up with the fact I can add video, graphics, build gnarley dynamic apps and not have to sweat a major load about disk space. We can thank the invention of
perpendicular recording back in 2005 for these advances in disk space.
Further, memory is cheap as well. We can quadruple memory today for the same price we paid 3-4 years ago.
And, I do not want to forget the low level infrastructure developers for their advances and optimizations in webserver and rdbms software.
Development has always been a field dominated by young bucks. But unlike us old geezers, they can't remember back to the prehistoric era when we were dealing with portable storage media that maxed out at 1.44mb, A hard drive of 40mb was the norm. You were envied by others if you had 80mb. And 512kb of memory was standard (like 2gb is today).
I am not just waxing nostalgic here. What I am trying to say is that back in that era, the hardware we had forced us to make technical design decisions that were germane to eaking out performance in the applications we developed. And, as a footnote to this, my technological forefathers would chuckle at me and say "sonny, you don't know what lack of resources are. We programmed on punch cards and had to run and debug our programs when we could actually get a time slot on the mainframe that took up an area the size of your house and had 1/1000 of the resources that you get today on that there iPod of yers".
My point is that I grew up in an era when resources were at such a premium we had to seriously consider optimization in our application development. Even today I still develop with this mindset.
Today we can throw hardware at a problem.
But, I do think performance is a key issue an application developer needs to factor into their code. Okay, programming has changed for many of us. I remember having to design databases where a table packed everything into a row that was 2k or less. Writing apps in C one would need to write queues and stacks and ponder using the verb register for which variable. Declaring a pointer against a pool of memory that was the size of a thimble of water compared to todays 4-person hot tub.
But, still there are many things todays developer can do to optimize business apps. For one, minimize disk access. I see programs all the time that invoke multiple queries or stored procedures when only one is needed. Also, there is also nothing wrong with using integers as foreign keys in database design. Using quit loop functions once you have what you need can be a time saver.
A lot of optimization of course depends on matters like how many concurrent users your app will have to contend with. And, what is the intended use of the app is another prime consideration. There are many more.
I would be the first to admit that if you are writing an app that will only have a few users saving 10ms access time through varying your coding practices is not worth anyones time.
I remember many years ago a peer looking at a Perl program I had written and giving me an obtuse (in my opinion) way of doing the same thing. Well, first off, my program was a command line batch program that was going to be run maybe once a week at the most by one programmer on their pc (it was a code generator. I won't bore you with the details). But I was curious. So, I went back to my desk and created two scenarios. One coding my way, one the way I thought was obtuse, I ran them in loops executing 1 million times using a hi-res timer to benchmark performance. And, the dude was right-on, his was faster. By about 15 milliseconds total under my test scenario.
For my intended use it made zero sense to change the code to a coding convention that seemed very foreign to my way of thinking. Plus, I thought, hey the single end user of my app may not have time to go out and have a beer and get back fast enough with time savings like that. We must always be considering the needs of our customer.
On the other end of the spectrum, during the era when client server apps were in-vogue I saw some fat client apps written that would create and dump (errr...I mean deliver) 3-5mb of data to the client in one fell swoop, taking a couple of minutes to load. Hmmmm, I remember thinking the customer will really dig that. And, this story is from a time frame when 16mb was a standard physical memory footprint.
Today is such a cool time to be computing in. One can consider true multimedia solutions in their design and not have to worry about needing to sell the customer a cray to host the app. The emergence of mobile devices (even with their evil proprietary languages) is way cool as well. Streaming video still impresses me.
And, knowing within ten years that someone will figure out how to mainstream a monitor you can see in your glasses, and a virtual full sized keyboard to type on that can be tied to your handheld. I can just picture someone on the bus coming home from work (that would not BE ME!!!) madly typing away some flame email on a keyboard none of us can see. Man, I can't wait for that day :-)
5 years ago... Well, my current computer, I am about to swap out this friday has 80gb
15 years ago... My work issued computer was a 486dx-32mhz with a 120mb drive. Whohooo, or as they say in the Bronx, "fuuugettaboutit"
I remember back in the late nineties building web content and worrying about what impact adding a 3k or 4k graphic would do. Today I am having a hard time keeping up with the fact I can add video, graphics, build gnarley dynamic apps and not have to sweat a major load about disk space. We can thank the invention of
perpendicular recording back in 2005 for these advances in disk space.
Further, memory is cheap as well. We can quadruple memory today for the same price we paid 3-4 years ago.
And, I do not want to forget the low level infrastructure developers for their advances and optimizations in webserver and rdbms software.
Development has always been a field dominated by young bucks. But unlike us old geezers, they can't remember back to the prehistoric era when we were dealing with portable storage media that maxed out at 1.44mb, A hard drive of 40mb was the norm. You were envied by others if you had 80mb. And 512kb of memory was standard (like 2gb is today).
I am not just waxing nostalgic here. What I am trying to say is that back in that era, the hardware we had forced us to make technical design decisions that were germane to eaking out performance in the applications we developed. And, as a footnote to this, my technological forefathers would chuckle at me and say "sonny, you don't know what lack of resources are. We programmed on punch cards and had to run and debug our programs when we could actually get a time slot on the mainframe that took up an area the size of your house and had 1/1000 of the resources that you get today on that there iPod of yers".
My point is that I grew up in an era when resources were at such a premium we had to seriously consider optimization in our application development. Even today I still develop with this mindset.
Today we can throw hardware at a problem.
But, I do think performance is a key issue an application developer needs to factor into their code. Okay, programming has changed for many of us. I remember having to design databases where a table packed everything into a row that was 2k or less. Writing apps in C one would need to write queues and stacks and ponder using the verb register for which variable. Declaring a pointer against a pool of memory that was the size of a thimble of water compared to todays 4-person hot tub.
But, still there are many things todays developer can do to optimize business apps. For one, minimize disk access. I see programs all the time that invoke multiple queries or stored procedures when only one is needed. Also, there is also nothing wrong with using integers as foreign keys in database design. Using quit loop functions once you have what you need can be a time saver.
A lot of optimization of course depends on matters like how many concurrent users your app will have to contend with. And, what is the intended use of the app is another prime consideration. There are many more.
I would be the first to admit that if you are writing an app that will only have a few users saving 10ms access time through varying your coding practices is not worth anyones time.
I remember many years ago a peer looking at a Perl program I had written and giving me an obtuse (in my opinion) way of doing the same thing. Well, first off, my program was a command line batch program that was going to be run maybe once a week at the most by one programmer on their pc (it was a code generator. I won't bore you with the details). But I was curious. So, I went back to my desk and created two scenarios. One coding my way, one the way I thought was obtuse, I ran them in loops executing 1 million times using a hi-res timer to benchmark performance. And, the dude was right-on, his was faster. By about 15 milliseconds total under my test scenario.
For my intended use it made zero sense to change the code to a coding convention that seemed very foreign to my way of thinking. Plus, I thought, hey the single end user of my app may not have time to go out and have a beer and get back fast enough with time savings like that. We must always be considering the needs of our customer.
On the other end of the spectrum, during the era when client server apps were in-vogue I saw some fat client apps written that would create and dump (errr...I mean deliver) 3-5mb of data to the client in one fell swoop, taking a couple of minutes to load. Hmmmm, I remember thinking the customer will really dig that. And, this story is from a time frame when 16mb was a standard physical memory footprint.
Today is such a cool time to be computing in. One can consider true multimedia solutions in their design and not have to worry about needing to sell the customer a cray to host the app. The emergence of mobile devices (even with their evil proprietary languages) is way cool as well. Streaming video still impresses me.
And, knowing within ten years that someone will figure out how to mainstream a monitor you can see in your glasses, and a virtual full sized keyboard to type on that can be tied to your handheld. I can just picture someone on the bus coming home from work (that would not BE ME!!!) madly typing away some flame email on a keyboard none of us can see. Man, I can't wait for that day :-)
Saturday, August 21, 2010
Developing a Web Development Framework - Walking the Walk
(Actual Written on 7/31/2010)
Software architecture and design is and has always been important to me. A big part of of this is implementation. It is easy to code a hack. We all do it. Time constraints, money, whatever the reason making a hack is easy to do.
After one does enough gigs they should come to a realization that doing the same work over and over again can be simplified or organized in a way that makes life easy, shortens the development cycle, is easy to manage and maintain. With my alchemy web development framework this is a realization I am trying to achieve.
My main design goal regarding implementation is really founded on two main objectives.
1. Be able to quickly implement alchemy anywhere.
2. Easily update an existing alchemy implementation without breaking anything.
Alchemy is in alpha, but is already becoming a monster. The application framework alone is seventy directories and over 550 files. To makes matters just as interesting I host the framework on different posix servers, but my development environment is Windows.
I have just created a Sourceforge project to manage the builds. Right now I have started to move the content and documentation into my developergeekresources.com site. I am having to implement alchemy there as well in order to offer up the variousd templates and toycode examples.
Ultimately I want the alchemy framework to replace my four active websites. Replacing an existing legacy site is easier said than done. I want to make adding new content to them even easier than it currently is.
If you cannot effectively use your own stuff why would you expect anyone else to use it? Right now what I have is still rugged in certain areas. The only way to really see if you are on the right path is to use your own stuff. Too often software has become a modern factory where people that are producing the software never have to use it. Most do not even have to implement it, so they can be totaly insensitive to that discipline as well. A buddy of mine likes to say "fish or cut bait". I like to rephrase it to say "cut bait, then go fishing".
I must admit I am so busy my inclination and nature is to just slam something in and get back to work. Not such a good strategy if you are trying to build an entire framework. You gotta always be thinking about shit like "what will that change do to all the existing stuff?" and "how is that going to be able to implemented in a way that is extensible a year from now to include all the cool new stuff myself or someone else has conjured up that I have yet to even think about or consider.
Software architecture and design is and has always been important to me. A big part of of this is implementation. It is easy to code a hack. We all do it. Time constraints, money, whatever the reason making a hack is easy to do.
After one does enough gigs they should come to a realization that doing the same work over and over again can be simplified or organized in a way that makes life easy, shortens the development cycle, is easy to manage and maintain. With my alchemy web development framework this is a realization I am trying to achieve.
My main design goal regarding implementation is really founded on two main objectives.
1. Be able to quickly implement alchemy anywhere.
2. Easily update an existing alchemy implementation without breaking anything.
Alchemy is in alpha, but is already becoming a monster. The application framework alone is seventy directories and over 550 files. To makes matters just as interesting I host the framework on different posix servers, but my development environment is Windows.
I have just created a Sourceforge project to manage the builds. Right now I have started to move the content and documentation into my developergeekresources.com site. I am having to implement alchemy there as well in order to offer up the variousd templates and toycode examples.
Ultimately I want the alchemy framework to replace my four active websites. Replacing an existing legacy site is easier said than done. I want to make adding new content to them even easier than it currently is.
If you cannot effectively use your own stuff why would you expect anyone else to use it? Right now what I have is still rugged in certain areas. The only way to really see if you are on the right path is to use your own stuff. Too often software has become a modern factory where people that are producing the software never have to use it. Most do not even have to implement it, so they can be totaly insensitive to that discipline as well. A buddy of mine likes to say "fish or cut bait". I like to rephrase it to say "cut bait, then go fishing".
I must admit I am so busy my inclination and nature is to just slam something in and get back to work. Not such a good strategy if you are trying to build an entire framework. You gotta always be thinking about shit like "what will that change do to all the existing stuff?" and "how is that going to be able to implemented in a way that is extensible a year from now to include all the cool new stuff myself or someone else has conjured up that I have yet to even think about or consider.
Thursday, August 12, 2010
Open Source and Google
The other day I was building a portion of an app that dynamically generated charts based on the results of a query. Worked like a champ in my local development environment where the server and the browsers I use both reside on my laptop.
Then I uploaded it to my test server and the chart did not update without having to refresh the browser. Of course, the browser was caching the image and even though the data was changing, the file name remained the same. What to do?
Get the end users to lower there expectations?
Call it a WAD and modify the requirements?
Read a manual until I went into a coma?
No
I Googled: php image not refreshing ...
... resulting in a link on page one that took me to the solution. (and for that, they deserve a link.)
http://forums.powweb.com/showthread.php?t=60177
Generally, I can find an answer to my questions through Google spending only five or ten minutes. One of the most profound and unheralded innovations in the discipline of computer science is the marriage of search engines and open source software.
Sure beats the days of being dead in the water for days or weeks, getting bounced around and listening to muzak on support lines, and then having to come up with convoluted workarounds because you could not find a solution.
Then I uploaded it to my test server and the chart did not update without having to refresh the browser. Of course, the browser was caching the image and even though the data was changing, the file name remained the same. What to do?
Get the end users to lower there expectations?
Call it a WAD and modify the requirements?
Read a manual until I went into a coma?
No
I Googled: php image not refreshing ...
... resulting in a link on page one that took me to the solution. (and for that, they deserve a link.)
http://forums.powweb.com/showthread.php?t=60177
Generally, I can find an answer to my questions through Google spending only five or ten minutes. One of the most profound and unheralded innovations in the discipline of computer science is the marriage of search engines and open source software.
Sure beats the days of being dead in the water for days or weeks, getting bounced around and listening to muzak on support lines, and then having to come up with convoluted workarounds because you could not find a solution.
Friday, July 30, 2010
Computer History - Computer Technology and Prices from 1992
I was playing around with my 64gb iPod Touch and it struck me how far technology has come. I thumbed through an old "Computer Shopper" from November 1992 that I have saved. It brought back some fond memories from a time when I a) still had hair, b) was broke raising two young kids, c) used to salivate at all the cool new technology I could not afford.
For instance ...
Yesterday (July 29th, 2010) I bought a 750gb external travel drive for $99. Check out these prices for Seagate drives from November 1992...
A 245 megabyte drive cost $563 dollars! Too rich for my blood.
Technology sure has come a long way in terms of what you get for the dollar.
For instance ...
You could buy a state of the art 486DX system for a mere $2895. To put that amount in perspective, that would be a little over $4500 in todays dollars.
I was still about six months away from upgrading my 286 PC with a 10mb (thats right, 10 megabyte) to a screaming 486sx Packard Bell with a 40mb drive. I recall keeping a spiral notebook adding up all the software on my 286 so I could keep track of how much space I had left for data. Back in those days hard drive space was scarce. One used their 5-1/4" or 3-1/2" floppy drive as the primary means of storing data. The 3-1/2 floppies were great as you could buy double density disks that held a whopping 1.44mb. Today I use a 4gb flash drive I paid $12 for.
I also recall marveling when I bought my 486sx in mid 1993 at how much the prices had come down. It only cost me $599 for the computer and a 40mb drive and 512k (yes that is kb not mb) of memory.
That was no small upgrade for me as it allowed me to start using Windows. On my 286 I used DR DOS as my operating system and a cool DOS task switcher named Deskview. I actually did have a copy of Windows 286 (bet most of you never heard of that version.) It did not hurt working down the street from Microsofts main campus in Redmond that back in the day only consisted of a few buildings.
That was no small upgrade for me as it allowed me to start using Windows. On my 286 I used DR DOS as my operating system and a cool DOS task switcher named Deskview. I actually did have a copy of Windows 286 (bet most of you never heard of that version.) It did not hurt working down the street from Microsofts main campus in Redmond that back in the day only consisted of a few buildings.
Yesterday (July 29th, 2010) I bought a 750gb external travel drive for $99. Check out these prices for Seagate drives from November 1992...
A 245 megabyte drive cost $563 dollars! Too rich for my blood.
Hah! You can buy a low end laptop today for $563. Here is a Laptop wannabee from 1992...
Five grand for a 486 with 4mb or ram and a 120mb hard drive. This was a rockin' machine back in 1992.
In 1992 we were only a decade into the PC revolution. Thanks to Intel and Microsoft and IBM we had come a long way from the PC-AT and PC-XT era. I remember working on PC's in the 80's that had no hard drives. They would come with dual 5-1/4" floppy drives. In one drive you would put your software disk (like Word Perfect) and in the other floppy you would have your data disk. You could store 360k worth of data on these floppies. If you had a double-density drive and disk you got a massive 720k of storage.
CD ROM (RO standing for Read-only) was starting to be sold under the hearding multimedia in later 1992. You could buy a CD Rom Drive for a mere $350. You could store 650mb of data on them. If you wanted a CD Drive that you could write to it would cost you at least $10,000 and these drives were not available through retail. In fact, even CD ROM drives were not even standard equipment in high end PC's in 1992. I remember getting so stoked at 650mb of storage. "Wow", I thought, you could store a whole library on one disk. Now we carry devices in our pocket that allow us to make phone calls and can hold movies and our entire music library. We can watch movies and televsion streaming to our laptops across the internet.
I thought this would be an interesting post from an old technology geezer for some of you younger folks that are not old enough to remember the early days of the PC revolution.
CD ROM (RO standing for Read-only) was starting to be sold under the hearding multimedia in later 1992. You could buy a CD Rom Drive for a mere $350. You could store 650mb of data on them. If you wanted a CD Drive that you could write to it would cost you at least $10,000 and these drives were not available through retail. In fact, even CD ROM drives were not even standard equipment in high end PC's in 1992. I remember getting so stoked at 650mb of storage. "Wow", I thought, you could store a whole library on one disk. Now we carry devices in our pocket that allow us to make phone calls and can hold movies and our entire music library. We can watch movies and televsion streaming to our laptops across the internet.
I thought this would be an interesting post from an old technology geezer for some of you younger folks that are not old enough to remember the early days of the PC revolution.
Technology sure has come a long way in terms of what you get for the dollar.
Subscribe to:
Posts (Atom)