Response to: Quality vs Quantity and the Ceramic Art Students

I’ve seen the following snippet copied and pasted quite a few times on Hacker News:

The ceramics teacher announced on opening day that he was dividing the class into two groups. All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality. His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the “quantity” group: fifty pound of pots rated an “A”, forty pounds a “B”, and so on. Those being graded on “quality”, however, needed to produce only one pot – albeit a perfect one – to get an “A”.

Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the “quantity” group was busily churning out piles of work – and learning from their mistakes – the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.

From: https://blog.codinghorror.com/quantity-always-trumps-quality/

Of all of the Hacker News comments that I’ve seen that quoted it: They’re attacks against others who claim that you should consider quality when developing software. The problem I have with the claim and the quote is that it completely misses the point. The context of the quote involves unskilled learners that are trying to build an understanding and get practical knowledge for the subject. Within that context, quantity has a lot of value. The students are developing hands-on learning, building muscle control/memory, and are applying their knowledge.

Another issue I have with the quote is that it never states what this quality was. Was it still at the student level? Was it sellable?  Also, the quote doesn’t account for: What did the students learn? What would they do differently? The goals of education are different from the goals outside of education.

As a professional software engineer, you are paid to produce professional quality software. Completing tasks to show that you completed task demonstrates that you can close tickets. It produces no evidence that you can deliver quality software. That’s the same with any engineer. Build 20 bridges, in which a couple fall, you’re going to be considered to be a bad bridge engineer. Also, all your previous work is going to be put in question.

I realize that the knee-jerk response to the previous statement is: “But Steven! Over engineering is bad, the business is going to fail because it can’t deliver/you can’t deliver like that/you have to have the experience to produce stuff and that comes from quantity.”  To address those see:

  1. Business is going to fail: There was poor planning and over promising done here many times over. Blaming the tech team for over-promising is trying to pass the blame along. Software takes time, we’ve got decades of experiences to illustrate this. If quantity were the ultimate winner, we wouldn’t have unrealistic deadlines.
  2. You can’t deliver like that: You can. You have to start with quality early, do the appropriate planning, and write software in a way where you can reuse existing code. Code reuse of quality code gives exponential returns. Sloppy code gives exponentially losses. You can’t turn around quality code later after most of the code is based on sloppy, unreliable, and untrusted code.
  3. You have to have the experience to produce stuff, and that comes from quantity: That statement is a bit of a strawman. Write the software, test it, and ship the code. Just don’t try to ship everything and then go back to try to rewrite everything because you didn’t do your due diligence the first time. Also, I never made the statement claiming there was no forgiveness for misunderstandings before. When you go back, and you went with quality first, the reason you went back to the code from before is that you realize that there was a misunderstanding. (Not because you just wanted to finish it early). The extra benefit is that you learned something.

Notes: Using Betamax in Grails (2.4.x)

Betamax is a great tool that records the output of a web service, and serializes the full request to a file. Once the request has been recorded, it can be played back in a test. This comes without any setup on your HTTP requests. This means that your system test no longer has a need to be connected to a working service, or has to be mocked based on a hand created mock.

Since the 2.3.8 release of GRAILs, there have been a few quirks to getting Betamax to work in your tests.

A few things need to be considered when using Betamax with Groovy Grails:

 

  1. Conflicting dependencies with the build process and SnakeYML
    https://github.com/robfletcher/betamax/issues/153
  2. The configuration file isn’t respected.
    Use the rule:

    @Rule
     Recorder recorder = new Recorder(
            tapeRoot: new File(BuildSettingsHolder.settings?.baseDir, 'test/resources/tapes'),
           ignoreLocalhost: true,
          sslSupport: true)
    //The sort property isn’t set: set this up in the test setup
    TapePropertyUtils.metaClass.sort = { Set properties, List names ->
    new LinkedHashSet(properties.sort(true, new OrderedPropertyComparator(names)))
    }
    
  1. Use Groovy to rewrite your RestClient

    def meetupBase = new RESTClient(url)
    BetamaxRoutePlanner.configure(meetupBase.client)
    meetupBase.client.removeRequestInterceptorByClass(ContentEncoding.RequestInterceptor.class)
    meetupBase.client.removeResponseInterceptorByClass(ContentEncoding.ResponseInterceptor.class)
    meetupBase
    
  1. Location of tape files in the project
    ~/ProjectFolder/test/resources/tapes
  2. Note about the hostnames host reference. (All of the tapes are based on the mocked name, so be prepared to modify this if need be.)

 

Things I’ve been Learning This Week (13 April 15)

Just as a quick summary of the things I’ve been up to this past week:

  1. Reading:
    1. The Last Girlfriend on Earth (Fiction)
    2. Grails In Action-  2nd Edition (Great book btw) [I’ve read the first one, and like it. I’m loving this edition as that it’s more recent and includes the new toys that the later Grails and Groovy editions offer you]
    3. Soft Skills (Also a great book)
  2. paramValidFields within generated Grails tests is meant as something you’re supposed to implement yourself. (This intended to create params that are valid.)
  3. The Fixture’s plugin for Grails has a few quirks
    1. All of the fixtures, unlike in Ruby with the equivalent, are based on a DSL in Groovy (so you have more Groovy files in your projects, and those are compiled in)
    2. The Fixture’s plugin intentionally tries to create valid domain objects (based on your set attributes)
  4. The fairly ubiquitous class Expando is something you can use yourself. It basically allows for you to dynamically define and expand your class (as you would with Python or Ruby)
  5. DropWizard: This has been a little bit of a challenge to get setup and working. However from the looks of it, it doesn’t look too bad.
  6. Chipotle isn’t horrible, its also not horrible. (I’ve been a bit resistant as that it’s a chain and is owned by McDonalds).  On that front I’m still waiting for Nandos to come to Chicago.
  7. How to use a Mac. As a linux person: it’s disappointed me quiet a bit> It’s ridged, not so user friendly, for my purposes, and it has some strong unforgiving opinions about how you’re to use a computer. Would I buy one? Nope.
  8. Using Hibernate via the JPA Annotations (I’ve always depended on the HBM and the Generation afforded).

First Thoughts: “Coders at Work” by Peter Seibel

I’ve just started to read the book Coders At Work. The book is a nice, recent collection of interviews from many big name developers. I’ve read other developer interview books before, but this one sticks out in an unusual way: with most “interview” books, the interview is either completely boring or incredibly interesting. In Coders At Work, the interviews have varied between amazing and neutral. I haven’t gotten to a bad interview yet.

A few things jumped out at me and made me think. Jamie Zawinski’s interview made me wonder about the value of non formally-educated developers in “today’s market.” Brad Fitzpatrick’s interview reminded me of the “I’ll build everything,” but you “must know everything” attitudes. Douglas Crockford’s interview didn’t inspire me, but it did make me consider other issues within software development.

Jamie Zawinski’s interview was an amazing conversation about a guy who has many interests in learning and doing work. He is a self taught LISP developer who can occasionally get very opinionated. I found his work experience with Netscape fascinating. As a user of the early versions of Netscape, I never knew all of the politics or construction going behind the scenes. I also found it technically intriguing that the pre-3.0 mail reader within Netscape was not written in C++. I have a lot of respect for Mr. Zawinski for being able to identify a potential bias of his – he appeared very introspective when asked about hiring new developers. He understood that he could distinguish people that he could find reputable, but not those who would make good candidates.

One of the things that struck me as a bit off-putting about Mr. Zawinski was his rejection of automatic unit testing. I feel that if it was made as easy in the 90s as it is today, software would be VERY different today.

Brad Fitzpatrick’s interview left me with mixed feeling about the guy. I’m not sure if he is a guy you would want to work with, however he sounds like the kind of guy that you would want to share war stories with over drinks. He has worked on many interesting projects, mainly LiveJournal, and is one of the early “Growth Hackers {http://en.wikipedia.org/wiki/Growth_hacking}.” I like his recommendation that you should spend some time in reading other’s code. He fights the immediate urge to ignore others’ code and his approach sounds different from what I had expected: I expected his approach to making suggestions on other people’s code would be antagonistic. However, it was described as the following:

  1. Code copies are distributed to the audience – in digital and paper form

  2. The developer presents their code line by line

  3. Q&A time

  4. Suggestions / feedback from the audience

This struck me as different from my experience where code reviews tend to be either technically or personally antagonistic (or both). This approach was more similar to proofreading a paper you just made or audience-testing a book you just wrote.

The two things that really put me off about Mr. Fitzpatrick was one of the questions he asks in interviews, and the other is the insistence of knowing everything. Mr. Fitzpatrick’s “famous” interview/programming question was a recycled question from his “AP CS” exam. The question is to write a class that handles large number arithmetic (rewrite BigDecimal). It appears that he uses his previous experience as a baseline for evaluation. I also got the feeling that it is a way for him to “show superiority over others” (over something he did many years ago a high school student). Second, he is incredibly insistent over knowing everything about how every layer works. He ranted against high-level language developers because they didn’t know that a specific way of polling may not work on a specific deployment machine. He even ranted over those who deployed towards a VM because the VM’s “virtual”->native OS/hardware has been abstracted. I feel that in 98% of the cases he’s picking up pennies in front of a steam roller.

I was not very thrilled with Douglas Crockford’s interview. Primarily because it dealt with Javascript, it was a little too high-level for my taste. During the reading of this interview, my mind went back to Mr. Fitzpatrick’s interview. It made me wonder and realize about how you find the “best” tools. I find it incredibly difficult to keep afloat of all the languages and tools available. Recently, for example, I just learned how – and why – Git, Jenkins (plus the automated unit/lint/reporting/checkstyle plug-ins), and deep Maven knowledge are really good things to know if you’re developing in Java.

When new languages, tools, and frameworks come around, I love to read about them and learn how they work (if they’re useful and interesting enough). However, time is limited: how do you identify the tools that would solve the most pressing need you have?  Prior to Jenkins, I built everything via an IDE. Why would I need an automated build tool? I’m the only developer on the project. Prior to Git, I used Subversion – it did everything I needed. Why would I want to make “sub-commits”? Prior to Maven, why would I want to have the build tool automatically deploy the WAR file or require that all unit tests pass before generating an executable? (I’m running unit tests all the time anyway.)

Later it made me think about the code reading suggestion and I realized: I’m not very happy with the code review tools I know about. ReviewBoard looks nice, but that is for only for Ruby. Should I write my own? Where are the existing tools for Java (which can also integrate with Maven and Jenkins)? Are the tools good? Are there others out there that have solved this issue? Is it worth setting up a code review tool just for this? This are questions I’m not sure how to answer.

Overall, I really enjoyed that this book goes over many topics – personal projects, interview questions, and famous debugging stories. I do occasionally enjoy a story bragging how the developer’s language or tool was miles ahead. However after reading about their accomplishments in a serial fashion, it just gets old. Perhaps interspersing their accounts in a more conversational form would have made this book more interesting, and easier to recommend.

Similarities of the individuals who were interviewed:

  1. All have a strong focus on one particular project

  2. Each interviewee has worked in many companies

  3. None of them focused on the reputation of the company they have worked for

  4. All have interesting debugging stories

Pastebin for HTTP Requests.

Have you ever had the need to verify an HTTP request? If so the tool “RequestBin” is the way to do it. It is the pastebin of HTTP Requests. When you use the site, it’ll give you a temporary custom URL to post whatever you need to, and it will display the contents when the request comes in.

This is awfully handy for testing Web hooks from Gitlab and Github.

Apache Wicket [In Action]: A Review and How It Relates to the Java World

Java is a great tool for creating software. It is well designed, modular, has a wide array of platforms that it can run on, performs well, it’s very extendable, and it has a large community with lots of support. However, it’s support for websites and related services is severely lacking. It’s bad enough that frameworks that extend the existing infrastructure have massive pitfalls that you later discover.

For the most part Apache Wicket solves many of the web related issues that J2EE (JSP) has. If you are to follow the prescribed way of doing things, it can actually be quite pleasant. However, there are a few thorny patches with Wicket. I will get to those later.

Wicket In Action like a marriage. During the honeymoon, everything is great. Everyone is happy, and then later discontent grows and things go up and down. However, unlike a marriage, you don’t really get an ending. This is a rather good way to end things, but some of the lesser parts of the book were rather disappointing. One of the big selling points of Wicket is that it is a framework that assumes that the developer has already prototyped the pages in HTML prior to starting with Wicket [The pages are adapted into Wicket-ized and previewable pages]. This upside was enough to ignore the placement of the HTML files, in the class path rather than the web resources section.

I jumped into this book with lots of enthusiasm after reading the introduction. I even bore through some of the non-stated setup issues in the book. The book starts off by creating a project, from scratch, however I went the Maven route (which I discovered is the better way to go). The book mentions maven, but it doesn’t mention how to build your application or to generate a project. I believe that I went the correct way because Maven helped to setup all of the application servers and the folder structures. The book started by having the user to jump in, examine a few code segments, and then to start on a sample (full featured) e-commerces application. The store was oddly pleasant; the goal was to sell cheeses online. The application started from a few sample view pages, it went on to creating a reusable shopping cart, and finally on to a membership mechanism. This is a very straightforward and to-the point way of starting a new framework. It’s already addressing the needs of the majority of its audience.

Another nice thing to point out about the introduction is that it did not try to cover all of the material at once. It would frequently describe what you were doing, but would mention the chapter where the concepts were explained in depth later on. Something that pleased me was that the code listings did not include a listing number. They were place in the correct location of the text. After you’re done with the sample application, you should be quite proud of yourself. This is similar to your first website.

However, the book got a little disappointing when describing the more detailed interworkings of Wicket: sessions, bookmarkable pages, and layering/rendering. The book improves when it gets to the Ajax functionality and a brief mention of dependency injection and Wicket. The book gets a little rough in the Spring through Hibernate sections and then better in the testing section. The book ends in a rather low note on SEO, production configuration, and JMX. If I had known more about JMX, I would have probably had a better opinion of the ending.

Overall I am not sure if I can say that the less than stellar sections of the book were entirely the authors’ or the book’s fault. It quite possibly may be the technology’s fault.  I would strongly recommend the book if you are new to Wicket.

Lastly, here are some direct tips that I had to discover on my own that helped out a lot:

 

A Few Signs That Your Project May Be In Some Serious Trouble

  1. Lack of leadership in developer operations
  2. A lack of reporting of improvements in code reuse, coverage, test cases, and stability.
  3. Underutilization of frameworks and libraries used within the project.
  4. Lack of disposable runtime environments [for development and testing].
  5. Build times are unknown.
  6. Long and detailed documentation is required to build your product.
  7. Dead code is left in the code base to rot for years.
  8. Bugs are being reproduced as unit tests.
  9. Third Party dependencies are not being upgraded.
  10. Code is badly organized. Models are placed anywhere, packages aren’t organized, etc.

The Pitfalls of Testing

For the most part, testing is a great thing that is a surefire way to improve software quality. However, I’ve found that there can be three minor pitfalls when it comes to testing. I’m sure there are more, but these are the ones that I’ve experienced.

When running the tests, your current runtime environment is based off the testing class path. This means that when you are using refactoring, the classes that the test cases are contained within can appear to be valid packages within your application. For example, when recursively searching for a listing of classes by using reflection, the test classes will show up. This is an important thing to note, and to remove the test classes in your cases, when you’re confirming the results of this process. In addition, it is possible to use reflection to see and initiate test cases from the application logic, something that should not be a valid operation in unit testing. For example: If you had a Factory class to initiate only the classes within your JAR file, you could pass in a reference to a test class, and it would initiate it. This would produce undesired behavior.

IDEs consider test cases to be a usage of code. This makes it more difficult to spot dead code in your project. Dead code slows down production time, tends to be non-conforming with the current code base/philosophy, and may be used by an ill-informed developer later.

As of the current JUnit version (4.11), there is no way to disable a test and have a warning created for this state. The only two options for this are to let the unused test case fail intentionally, or to use the ignore annotation. Intentional test case breakages performed by an intentional fail method within the test case, or leaving the functionality broken. Unfinished test cases [which would warrant this situation], should not be left blank, they will show up as a passed test case. Empty test cases should be shown as an error, but are not. This StackOverflow question discusses the Ignore annotation more; however using @Ignore leaves unused test cases, which the developer is responsible for keeping up with. Having a broken test case inflates the failed test case count, and may cause valid test failures to be overlooked.

Don’t trust your Technical Documentation Staff to write Quickstart guides and Tutorials, Hire a Professional.

This is written with an audience of producers of software products/services in mind. Please do not claim that your documentation, used to introduce a new user/developer, is good enough to make the user/consumer (developer) proficient in your technology. Producers of software products [the developers, and organizations] are great at writing references to their products, however they generally are horrible at introducing new users to their product. When quick start guides, or introduction tutorials are written for products, the documentation tends to remain in the domain that the creators made. Solution: Hire a reputable technology writer. This will help you in a few ways:

  1. Gets a credible authority figure giving approval and backing to your technology
  2. Loses a creator bias
  3. The writer, like your audience, has an outside knowledge of other technologies
  4. They should not be influenced by the creators to write things that lack value to your actual users. (This assumes that sales and management cannot affect the contracted author’s work.)
  5. The users will understand the product better, and thus reduce undesirable support calls/emails.
  6. You will have professional documentation. The technical writer’s domain is to create attractive and easily consumable books, the software producer’s domain is not the same.
  7. From a business sense, the author is not a full time employee; they would remain as a short-term contract. Insulation from actual production is a good thing. This also helps the author as that it stabilizes their income streams.
  8. The documentation is written from a potential customer’s perspective
  9. It gives quick feedback on how easy the system, language, or product is to use, setup, troubleshoot, and possibly maintain.

Given these points, it would be easy to justify even a luxurious price on producing high quality introductions / non-referential documentation for your products. Based on these reasons, I conclude it would benefit the business professionals, developers, testers of the product, users, and the marketing department by separating the author him/herself from the general stress of the entire producing company.

“Wanted Java Developer” could you be a little more ambiguous?

I have a bone to pick with the industry I associate with. The tech industry struggles to clearly define the expectations of whatever is desired. This is pretty much a universal issue with the industry. There are always hidden requirements. Job listings are no different.

The bone I have to pick is with the titles/job requirements. One of the worst offenders of this is for a Java Software Engineer. Based on the availability of frameworks, and meta-frameworks, asking for a Java engineer is quite ambiguous. This could ask for a graphics developer, API designer, web developer, Computer vision expert [CV in java is possible with native libraries, trust me!], core libraries developer, or even a micro-JVM developer. The amount of variability with the language makes a generic “Java developer” title frustrating.

As a potential candidate, it is incredibly frustrating to see the “Java Developer” title. Some titles ask for experience in Spring, Hibernate, Solr, Lucene, JAI, J2EE, etc. It’s incredibly frustrating to go into an interview, where the listing asked for all of these and then only be grilled on the minute [rarely used] inner workings of Spring RMI. Just the Spring framework alone asks for a lot. Of all of the potential avenues of spring that one could master are: Message queuing, Web services, Roo, Security, Integration, Web flow, MVC, BlazeDS, Batch, Social, and mobile. That’s not even accounting for the frameworks that you can substitute between the pathways for Spring. I’ve worked with a late Spring 2.x and early 3.0, I was not even aware of the new BlazeDS, Batch, social, or even mobile options for the framework. Things change quite quickly.

Besides ranting, what is the purpose of this article? I believe that the Java title should be a bit more specific. If you want a Java developer that knows a lot of frameworks, still label it as a Java developer position. However, don’t expect him or her to know all of the inner workings: that is just silly. If your business involves multimedia display, request a developer that knows the Java 2D and 3D graphics, and maybe the JAI libraries. There is a lot there for practically any task you want to throw at Java, given you know which library to use.

Scientific or mathematical tasks, ask for a Scientific Java Developer. What libraries should they know? Colt, EJML, JAMA, etc.

Writing a Java API? Ask for a Java API Designer. Expect for them to list their favorite APIs, what works, what doesn’t, and why.

Web applications? Ask for a Java Web Developer. Maybe they should have experience with a SOAP framework, MVC framework, Play, GWT, basic Web skills, and maybe even a non-SOAP based webservice library [REST, Hessian/Binary]. Please don’t use the enterprise title unless you need someone who knows EJB, and/or ESB frameworks.

Moreover as an employer, be more specific the first interview/inquiry on what the job is asking for. If an job application asks for the common ORM Hibernate, how much should one know about it before applying for the position? Should they be able to know how to wire up beans to their DB analog? Should they know how write their own dialect for a new data source? With JUnit, should the applicant be expected to transition your current development methodology to TDD? Should they know how to extend the JUnit framework?

At this point, you should be getting the picture. Different tasks demand different skills. Specify what you’re looking for. Find people that can learn new skills and that are interested in the same problems you are. People who have an active interest can learn what is needed to get up to speed. If you go to a butcher and ask for meat, you’re always going to get what you asked for, but not what you were craving.

Addendum: This can apply to an Erlang, C++, Python, and most other developers. The role of Java currently has such a great demand, and such an ambiguous title, making this article a little easier to communicate.