My Favorite Programming Tools

As a programmer I am using almost every day a number of tools that help me achieve various programming tasks. I would like to share them here with the people interested and of course any feedback about other tools is greatly appreciated.

I use a number of utilities like:

• Notepad++  - this is a very popular open source text editor for fast editing and file comparison, I find it extremely useful and I use it almost every day. It comes with a number of pluggable components and it has a good community.
  More information at: Wikipedia Notepad++.
• Putty - I am using this utility mainly for remote access. PuTTY is a free and open source terminal emulator application which can act as a client for the SSH, Telnet, rlogin, and raw TCP computing protocols and as a serial console client.This is the place were one can find this tool: Download Putty. Here are some tips on setting up your Putty options. And here is a tool to wrap multiple Putty sessions in tabs: TTY Plus. 
• WinSCP (Windows Secure CoPy) is a free and open source SFTP, SCP, and FTP client for Microsoft Windows. Its main function is secure file transfer between a local and a remote computer. Beyond this, WinSCP offers basic file manager and file synchronization functionality. For secure transfers, it uses Secure Shell (SSH) and supports the SCP protocol in addition to SFTP. This is the place to find this utility: Download WinSCP
  
• VirtualBox (Oracle VM VirtualBox) is an x86 virtualization software package. I use this tool to create quick development testing environments. To obtain VirtualBox visit the VirtualBox website's download page.
• soapUI - is an open source web service testing tool for service-oriented architectures (SOA). I use this tool for service testing during development. This tool can be found at: Download soapUI
• SFK (Swiss File Knife - Windows cmd utility) - is a command line tool for daily tasks. Find and extract text in binary files, list dir tree sizes, filter and replace text, run an instant ftp or http server for easy file transfer, find duplicate files, join many text files into one, create and verify md5 checksum lists, run a command on all files, detab text, create hexdumps from files, trace contents of a tcp connection, find dependencies between files, print colored text to terminal, locate commands in the path, print last lines of a file, convert CR/LF, hex to binary, binary to source code, split and join large files, list the contents of all .zip, .jar, .tar, .gz, and .bz2 files. One can download this utility at: Download SFK
• QuickPHP - Useful utility for quick PHP UI mockups. It is very easy to install and configure. One can download it from here: Download QuickPHP.
• GIT - this is my favorite distributed revision control system; fast, simple to use and install.
• Eclipse - free development environment. I use this application for complex projects.
• JIRA - this is a great issue tracking system. 
• Confluence - great collaboration and wiki software. I use this application for all purpose documentation.
• TeamViewer - software used for remote control, desktop sharing, online meetings, web conferencing and file transfer between computers. This is a great tool. I use this tool for desktop sharing when I need to help members of my team.
• Dropbox - is a file hosting service that offers cloud storage, file synchronization, and client software. This is a great tool, it is free for 2GB storage and I use it as a virtual USB. I move files between many computers and systems that I use.

I am going to improve this list as I come over other tools that I use every day and I find very good.

Developing applications with OfBiz (Part 1)

I've adopted two months ago OfBiz framework (version 10.04.02) for the company that I am currently working for. We needed a fast eCommerce solution with a billing system.
I've chosen this solution after I've analyzed a large number of frameworks (open source and proprietary) - ERPs, CRMs and billing systems.
This framework was the most attractive to me as it had most of the functionality that we needed and the internal architecture was pretty nice and modern. It was easy to install the demo and play with it.
It provides support for typical three tier architecture applications: presentation (MVC), business logic and persistence. The design is very modular with components and applications. It is very easy to override, extend or even rewrite parts of the framework. It is very easy to plug-in other engines like; templating engine or workflow engine if desired.
I knew before I've started that this framework is huge (its data spans over 850 tables) and it will require a steep learning curve on my side. This framework was developed over 10 years and it is very complex. It had various redesigns at various stages.
I've started by trying to change the eCommerce application that comes with the framework as I hoped I can get some fast results (sort of hacking approach to show a quick prototype). This was such a bad idea; this application is so complex that unless you are familiar with this framework you will get lost. By just looking at the code and searching documentation I could not understand properly the entire mechanism. Besides, the default UI design and styles are so ugly, outdated and poorly implemented that it made me reconsider my approach.
My next approach was to read as much documentation as possible (of course I didn't have time to read all the existing documentation) and try to develop a very simple application by looking into the provided examples (cover implementation at all layers and try the engines provided with the framework). In the mean time I had prepared the requirements document, high level design, UI design together with my team mates.
This was a constant struggle as the documentation is not up to date, cannot be found in one place and some parts are older or newer than others.
I've found these tutorials which contain three hello world applications; none of them worked for me without struggle. The tutorials are otherwise good but one cannot download the code so it is easy to mistype some code and have many errors and exceptions (I've found this stupid). I could not even find blogs or comments on why various parts did not work, I was left on my own devices... After some days of struggle (almost 1 week) I was able to show to my team a simple application, far from being a perfect example of how we should implement in practice and less than I wanted to show them.
I've tried to implement services using OfBiz minilang. Although it is very simple to create a service if something goes wrong the errors are so cryptic that it is impossible to tell what the problem is. I also could not find much documentation so far. The service implemented in Java although are very verbose it is so simple to handle errors and to trace specific actions.
The entity engine is very nice, as it removes the need to work directly into the database. Any change to the database is done through a configuration file and a restart of the application. It sounds easy but now, instead of SQL language one has to deal with the specifics of the entity engine and is quite a steep learning curve.
I discovered some good documentation in regard to the framework architecture hidden into the actual code (that I recommend):

• Entity Engine
• Widget Toolkit Information 
• Service Engine Guide
• Text Translation

To be continued...

Predicting the future

"While we have sci-fi visions of room temperature superconductors like in the movie Avatar, the question still remains: How would the discovery of a such a material impact our everyday lives? How would the nature of warfare change? How would the global economy react? What are the cultural pros and cons of such a technological shift?"

I like this unorthodox answer:

"The most realistic answer, but not the one you want to hear, is: Nobody really knows.

If history teaches us one thing than it is that we are horrible at predicting the outcomes of anything major. In hindsight, we can "explain" things, but our predictions suck so badly, it's a surprise we haven't given up on the subject. And that's for both experts and non-experts.
Nobody came even close to predicting the impact of computers. Or electricity. People didn't think WW1 would become the slaughterhouse it did. There are refugees around the globe who are living in "temporary" shelters, waiting to return home because the conflict will surely be over any day now. Some of them have been waiting for a decade and more.
The real impact of this technology, as most, will most likely not be anything that anyone today predicts, but something that someone in the future comes up with that nobody thought of before. That includes the inventors. I don't think Graham Bell ever thought that "please turn off your mobile phones" would be a screen shown in these newfangled movie theaters that just came about in his time."

Are Microsoft, Apple, Facebook and Google evil?

This is an interesting question. What evil means when we are talking about huge corporations?
I believe this has to do more with how the large public perceive a certain corporation or a certain brand,  how large was the impact of the lawsuits, litigation, defamation, intellectual property etc.

I will try next to explore some of the "bad and ugly" from the most famous high tech companies which I believe can set each and every company in its own evil corner. Next are some well known facts:

1. United States v. Microsoft (1998 - 2006) - it alleged that Microsoft abused monopoly power on Intel-based personal computers in its handling of operating system sales and web browser sales. The issue central to the case was whether Microsoft was allowed to bundle its flagship Internet Explorer (IE) web browser software with its Microsoft Windows operating system. As a result of this trial Microsoft was not severely punished but it acquired the image of the ruthless monopoly.
2. Apple and AT&T Mobility antitrust class action - In the class action filed in 2008 against Apple, Inc. and AT&T Mobility In re Apple & AT&T Mobility Antitrust Litigation, the plaintiffs claim that Apple and AT&T illegally restrained competition, locked consumers into agreements with AT&T, and punished consumers if they tried to leave. In their complaint filed with the court, the plaintiffs alleged violations of the Sherman Antitrust Act, breach of warranty under the Magnuson–Moss Warranty Act, and other violations of consumer protection laws; the plaintiffs also alleged that these violations occurred when consumers were offered iPhones by Apple only if they signed a two-year service agreement with AT&T Mobility, but that unknown to consumers at the time of purchasing the iPhones, the two companies agreed to technologically restrict voice and data service in the aftermarket for continued voice and data services after the consumers' initial two-year service period expired, while at the same time continuing to share revenue stemming from provision of voice and data services to iPhone users.
3. U.S. Files Antitrust Lawsuit Against Apple, Hachette (2012) - The U.S. sued Apple Inc. (AAPL), Hachette SA, HarperCollins, Macmillan, Penguin and Simon & Schuster in New York district court, claiming the publishers colluded to fix eBook prices.
4. Google Possible ties to the CIA and NSA - in February 2010 Google was reported to be working on an agreement with the NSA to investigate recent attacks against its network. And, while the deal did not give NSA access to Google's data on users’ searches or e-mail communications and accounts and Google was not sharing proprietary data with the agency, privacy and civil rights advocates were concerned.
   In October 2004 Google acquired Keyhole, a 3D mapping company. In February 2004, before its acquisition by Google, Keyhole received an investment from In-Q-Tel, the CIA's investment arm.^[60] And in July 2010 it was reported that the investment arms of both the CIA (In-Q-Tel) and Google (Google Ventures) were investing in Recorded Future, a company specializing in predictive analytics—monitoring the web in real time and using that information to predict the future. And, while private corporations have been using similar systems since the 1990s, the involvement of Google and the CIA with their large data stores raises privacy concerns.
   In 2011, a federal district court judge in the United States turned down a Freedom of Information Act request, submitted by the Electronic Privacy Information Center. In May 2012, a Court of Appeals upheld the ruling. The request attempted to disclose NSA records regarding the 2010 cyber-attack on Google users in China. The NSA stated that revealing such information would make the US Government information systems vulnerable to attack. The NSA refused to confirm or deny the existence of the records, or the existence of any relationship between the NSA and Google.
5. Google Buzz - on February 9, 2010, Google launched Google Buzz, Google's microblogging service. Anyone with a Gmail account was automatically added as a contact to pre-existing Gmail contacts, and had to opt out if they did not wish to participate.The launch of Google Buzz as an "opt-out" social network immediately drew criticism for violating user privacy because it automatically allowed Gmail users' contacts to view their other contacts.
6. Apple iPod, iTunes antitrust litigation - The case In re Apple iPod iTunes Antitrust Litigation was filed as a class action in 2005 claiming Apple violated the U.S. antitrust statutes in operating a music-downloading monopoly that it created by changing its software design to the proprietary FairPlay encoding in 2004, resulting in other vendors' music files being incompatible with and thus inoperable on the iPod. Apple has de facto monopoly on the iPod music.
7. Google "don't be evil" - this corporate motto started as an albatross and ended up as a boomerang as the company slowly monopolized the online advertizing.
8. Facebook cooperation with government search requests - government authorities rely on Facebook to investigate crimes and obtain evidence to help establish a crime, provide location information, establish motives, prove and disprove alibis, and reveal communications even without warrant.
9. Microsoft censorship in China - Microsoft has been cooperating with Chinese government in implementing an Internet censorship. Microsoft has been criticized by human rights advocates such as Human Rights Watch and media groups such as Reporters Without Borders.
10. Labor Group Finds Poor Working Conditions at Apple Supplier Foxconn (2012) - An investigation by the Fair Labor Association into factories operated by Apple supplier Foxconn in China found poor working conditions and worker abuse, leading Foxconn to pledge it will make improvements.
11. Google Book Search settlement gives Google a virtual monopoly over literature - the Authors Guild -- which represents a measly 8000 writers -- brought a class action against Google on behalf of all literary copyright holders, even the authors of the millions of "orphan works" whose rights holders can't be located. Once that class was certified, whatever deal Google struck with the class became binding on every work of literature ever produced. The odds are that this feat won't ever be repeated, which means that Google is the only company in the world that will have a clean, legal way of offering all these books in search results.
    
    
    Now my final question here is; who is more evil than others?  
    My personal belief here is that profits and power are all that matters for a huge corporation.
    
    
    
    
    
    

Billing Open Sources Review

I needed to find an open source billing system for one of the projects that I am working on. I've investigated a number of current open source projects.
I was interested in basic functionality like: invoices, orders, customers, estimates, orders, services, pricelists, payment, discounts, currencies etc. I wanted to find a reliable open source project to use and to contribute.

Criteria used to analyze the applications was: open source software evaluation
 
I've looked at SIWAPP, AgileBill, Amberdms Billing System, Freeside, CitrusDB, Bamboo Invoice, Simple Invoices.

SIWAPP is a new PHP project in beta stages and IE is not recommended.

AgileBill has no documentation, only one active developer.

Amberdms does not support all platforms, very little information.

Freeside is a Perl project and it has a bad UI.

CitrusDB is a PHP project maintained by one developer and the coding standards are a joke.

Bamboo Invoice is a PHP project in beta stages, has little documentation  and I could not find the source code.

Simple Invoices this is a PHP project, it does not contains all the functionality I was looking for.

Most of the projects are written in PHP. They are meteoric projects; started with enthusiasm, not very professional and lingering through years.

I could find only one that passed my criterias: JBilling.

JBilling looked very promising from my initial analysis. This project started in 2006, it has a mature well established codebase. It has documentation and coding standards. It is a web application Java based and uses standard Java frameworks. It has a number of developers and the code shows sustained activity.
It contains about 100K lines of Java and SQL code, which means that this is a medium size project. It has a fair amount of comments into the code.
It supports all major browsers and it is OS and database independent. Can run in most of the contemporary application servers that can run with JTA or local transactions, it requires a Message Queue implementation.

I've downloaded and installed the application as documented. The installation was very simple as it required JRE and the instructions were good. I was able to start checking this product functionality in less than 10 minutes.
The UI is very simple and intuitive; I was able to find very easy all the features that I was interested in. This was an unexpected well designed UI for an open source.

My next step was to obtain the latest code and to try to compile and deploy. One thing caught my attention when I was reading the development instructions: it requires Grails 1.3.4 (released sometimes in 2010). The latest Grails version is 2.0.3 released on 3rd April 2012. The latest minor version from version 1 is 1.3.8 released on 29 March 2012.
I was not able to compile using the instructions and the reasons were:

• Could not find all the dependencies for Grails 1.3.4
• Upper versions of Grails were not supported
• The code has dependencies on JbillingAPI and JbillingAPIFactory which are supplied only with EE

Customer care functionality is only available for EE.
This is a Catch 22 situation: the application is open source but it has some crucial missing parts so you have to buy...

So much with the open sources billing systems...

Update May 2013:
JBilling was acquired by AppDirect see article. Since there was no code change.

My personal opinion is: there is no feasible open source billing system and one should check commercial billing systems with extensive API. Check my analysis regarding the commercial billing systems (it is coming soon...).

My Best Programming Resources Online

I have a number of online resources that I consult when I need to gain knowledge or I believe are useful, I have questions or I want latest programming news.
I use only free resources and tools that don't require registration.

• WIKIPEDIA - this is by far the best encyclopedia ever. When I need to find an entry level information about any subject this is the first place I start looking at.
• STACKOVERFLOW - great programming forum, very professional and well moderated. I did find in many occasions very good answers to my questions.
• DZONE - programming resources and latest programming news
• JAVA ALLTOP - Java programming news
• CYBERCITI- Linux blog
• JAVA LANGUAGE SPECIFICATIONS - all Java language specifications
• APACHE - Java open source projects
• APACHE MAVEN  - Apache Maven website
• DVLPRS - famous programmers blogs and tweets
• SLASHDOT  - technology related news website
• REDDIT - social news  website, I read mostly compsci
• CPLUSPLUS  - C++ online resource
• GOOGLE C++ GUIDE - interesting C++ guide
• DEV GURU- JavaScript, HTML, CSS, XML, Server-side reference
• W3C WEBED WIKI (this site has a self-teaching curriculum as well as reference material): HTML, CSS reference
• W3C SPECIFICATIONS: HTML4, HTML5, CSS, DOM Tech Reports reference
• DEVGURU- JavaScript, HTML, CSS, XML, Server-side reference
• OWASP PROJECT  - the open web application security project
• WIKI MOZILLA SECURE CODING GUIDELINES - web security guidelines by Mozilla
• DEVELOPER YAHOO SECURITY - web security guidelines by Mozilla
• PYTHON - Python official website
• INTRODUCTION TO ALGORITHMS - iTunes introduction to algorithms lectures from MIT University but one can find many free lectures related to programming
• POSTGRESQL - Postgres official website
• W3SCHOOLS - SQL tutorial and tutorials to most web technologies
• SQLOPTIMIZATIONSSCHOOL - tutorial on SQL optimization
• LDAP GURU - LDAP programming guidelines
• DOFACTORY DESIGN PATTERNS - design pattern tutorial
• GOOGLE CODE UNIVERSITY - sample course content and tutorials for Computer Science 
• OHLOH - software directory, combining community-driven content with a unique source code crawler that monitors up-to-date development activity. 
• FINDJAR  - JAR search engine
• DOWNRIGHTNOW - popular services status

Great search engines:

• GOOGLE - best search engine
• WOLFRAMALPHA - computational knowledge

I will add more resources here as I find them.

Scientific Calculator in Minecraft by a 16 year old

This is a  very interesting fact:

"New submitter petval tips another amazing Minecraft project: a functioning scientific/graphing calculator. "On a virtual scale, the functional device is enormous — enough so that anyone in the real world would become a red blot of meat and bone staining the road if they fell from the very top. Honestly, his virtual machine looks more like a giant cargo ship ripped from a sci-fi movie than a working calculator. Yet type your problem out on the keypad, and the answer appears on a large white display mounted on the side of the monstrous brick structure." The creator says it can do "6-digit addition and subtraction, 3-digit multiplication, division and trigonometric/scientific functions ... Graphing y=mx+c functions, quadratic functions, and equation solving of the form mx+c=0." We've previously discussed the creation of a 16-bit ALU in Minecraft."