Feeds:
Posts
Comments

Posts Tagged ‘software development’

Out of Context – Using Core Data on iOS 9.3 and 10.0

Posted in Computers, Software Development, tagged , , , on 09 Sep 2016| Leave a Comment »

iOS 10 ships on September 13th. Apple has changed their generated code for Core Data. It doesn’t compile if you set your deployment target to iOS 9.3. What’s a developer to do?

For a very long time now, Apple has be proud, and deservedly so, that their users are overwhelmingly keeping up-to-date with the latest releases of their operating systems. As such, Apple’s messaging to developers is to support the current and previous OS version. Most of the time, this isn’t a big deal. Sometimes, it is. With the latest changes related to Core Data, it’s a problem.

If you create a new Xcode project that will being using Core Data, you get some useful code bits in the AppDelegate. In Xcode 8, you get the following (comments removed for brevity):

lazy var persistentContainer: NSPersistentContainer = {
    let container = NSPersistentContainer(name: "coreme")
    container.loadPersistentStores(completionHandler: { (storeDescription, error) in
        if let error = error as NSError? {
            fatalError("Unresolved error \(error), \(error.userInfo)")
        }
    })
    return container
}()
 func saveContext () {
    let context = persistentContainer.viewContext
    if context.hasChanges {
        do {
            try context.save()
        } catch {
            let nserror = error as NSError
            fatalError("Unresolved error \(nserror), \(nserror.userInfo)")
        }
    }
}

This works great if you’re only going to build for iOS 10, but if you try to set your deployment target to iOS 9.3 as recommended, you’ll get the following error:

AppDelegate.swift:49:35: 'NSPersistentContainer' is only available on iOS 10.0 or newer

Well, that’s not good.

As users of Core Data know, operations using it use a database context. You can see it being used in the saveContext function above. Here we see that the context is a member of the persistentContainer. Since this doesn’t exist in iOS 9.3, we’ll have to get it some other way.

Let’s look at what used to be supplied with Xcode 7 (udpated to Swift 3):

lazy var applicationDocumentsDirectory: URL = {
    let urls = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)
    return urls[urls.count-1]
}()

lazy var managedObjectModel: NSManagedObjectModel = {
    let modelURL = Bundle.main.url(forResource: "bfoo", withExtension: "momd")!
    return NSManagedObjectModel(contentsOf: modelURL)!
}()

lazy var persistentStoreCoordinator: NSPersistentStoreCoordinator =
{
    let coordinator = NSPersistentStoreCoordinator(managedObjectModel: self.managedObjectModel)
    let url = self.applicationDocumentsDirectory.appendingPathComponent("SingleViewCoreData.sqlite")
    
    do {
        try coordinator.addPersistentStore(ofType: NSSQLiteStoreType, configurationName: nil, at: url, options: nil)
    } catch {
        let dict : [String : Any] = [NSLocalizedDescriptionKey        : "Failed to initialize the application's saved data" as NSString,
                                     NSLocalizedFailureReasonErrorKey : "There was an error creating or loading the application's saved data." as NSString,
                                     NSUnderlyingErrorKey             : error as NSError]
        let wrappedError = NSError(domain: "YOUR_ERROR_DOMAIN", code: 9999, userInfo: dict)
        print("Unresolved error \(wrappedError), \(wrappedError.userInfo)")
        abort()
    }
    
    return coordinator
}()

lazy var managedObjectContext: NSManagedObjectContext = {
    let coordinator = self.persistentStoreCoordinator
    var managedObjectContext = NSManagedObjectContext(concurrencyType: .mainQueueConcurrencyType)
    managedObjectContext.persistentStoreCoordinator = coordinator
    return managedObjectContext
}()

I’ve left out the saveContext function as it’s only distinction is where it gets the context from. Above, we see that the context is a separate variable.

Now without a doubt, the iOS 10 code is much simpler, but, if we want to be able to do the right thing and support both current and last version, we need a bit of a mash up. It’s a bit tedious, but you only have to build it once (hint, hint Apple). Here’s what I’ve come up with.

// MARK: - utility routines
lazy var applicationDocumentsDirectory: URL = {
    let urls = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)
    return urls[urls.count-1]
}()

// MARK: - Core Data stack (generic)
lazy var managedObjectModel: NSManagedObjectModel = {
        let modelURL = Bundle.main.url(forResource: modelName, withExtension: modelExtension)!
        return NSManagedObjectModel(contentsOf: modelURL)!
}()

lazy var persistentStoreCoordinator: NSPersistentStoreCoordinator = {
    let coordinator = NSPersistentStoreCoordinator(managedObjectModel: self.managedObjectModel)
    let urls = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)
    let url = self.applicationDocumentsDirectory.appendingPathComponent(modelName).appendingPathExtension(sqliteExtension)

    do {
        try coordinator.addPersistentStore(ofType: NSSQLiteStoreType, configurationName: nil, at: url, options: nil)
    } catch {
        let dict : [String : Any] = [NSLocalizedDescriptionKey        : "Failed to initialize the application's saved data" as NSString,
                                     NSLocalizedFailureReasonErrorKey : "There was an error creating or loading the application's saved data." as NSString,
                                     NSUnderlyingErrorKey             : error as NSError]
        
        let wrappedError = NSError(domain: "YOUR_ERROR_DOMAIN", code: 9999, userInfo: dict)
        fatalError("Unresolved error \(wrappedError), \(wrappedError.userInfo)")
    }
    
    return coordinator
}()

// MARK: - Core Data stack (iOS 9)
@available(iOS 9.0, *)
lazy var managedObjectContext: NSManagedObjectContext = {
    var managedObjectContext = NSManagedObjectContext(concurrencyType: .mainQueueConcurrencyType)    
    managedObjectContext.persistentStoreCoordinator = self.persistentStoreCoordinator    
    return managedObjectContext
}()

// MARK: - Core Data stack (iOS 10)
@available(iOS 10.0, *)
lazy var persistentContainer: NSPersistentContainer = {
    let container = NSPersistentContainer(name: creditsModelName)    
    container.loadPersistentStores(completionHandler: {
        (storeDescription, error) in
            if let error = error as NSError?
            {
                fatalError("Unresolved error \(error), \(error.userInfo)")
            }
        }
    )
    
    return container
}()

// MARK: - Core Data context
lazy var databaseContext : NSManagedObjectContext = {
    if #available(iOS 10.0, *) {
        return self.persistentContainer.viewContext
    } else {
        return self.managedObjectContext
    }
}()

// MARK: - Core Data save
func saveContext () {
    do {
        if databaseContext.hasChanges {
            try databaseContext.save()
        }
    } catch {
        let nserror = error as NSError
        
        fatalError("Unresolved error \(nserror), \(nserror.userInfo)")
    }
}

I’ve introduced databaseContext as the context variable to be used throughout. I think it’s more clear and now saveContext doesn’t have to care which OS you’re targeting. With this change in place, you can now safely target 9.3 and still build for 10.0.

Read Full Post »

Jumping Through Loops – Thinking Swiftly

Posted in Apple, Computers, Software Development, tagged , , , on 25 Aug 2016| Leave a Comment »

One of the challenges in learning a new language, be it human or computer, is learning to think in the language. Learning Swift is no different.

I’ve been spending the past several months working on the first iOS (iPhone) app that I’ll be releasing through the Apple app store. I’m writing it entirely in Swift 3, the latest version of the language. Yesterday, I was looking over the code to see if there were any places where I’d been thinking in a non-Swift fashion. This moment of reflection was triggered by my viewing of a very recently released class in Swift from one of the major online education sites. I was disappointed that the instructor was using an idiom that I knew was easily simplified in Swift. My disappointment made me wonder if I had similar issues.

The following is actual code from my upcoming app. I had originally created it’s core in C as an extensible example for teaching that language. The setup is as follows:

  • the map is an array of connections from each room
  • the number of connections from each room is fixed
  • we are explicitly tracking the number of a given type of hazard

We want a function that returns true if there is hazard in one of the connecting rooms. Here’s the original code:

var hazardNear = false
        
if hazardCount > 0
{
    for nextRoom in 0 ..< numberOfConnections
    {
        if hazardRooms.contains(exits[room][nextRoom])
        {
            hazardNear = true
                    
            break
        }
    }
}
        
return hazardNear

Pretty conventional C-esque stuff. The first thing that jumped out at me was the reliance on knowing the number of connections. Swift arrays are true collections. Let’s treat them as such.

var hazardNear = false
        
if hazardCount > 0
{
    for nextRoom in exits[room]
    {
        if hazardRooms.contains(nextRoom)
        {
            hazardNear = true
                    
            break
        }
    }
}
        
return hazardNear

When viewed this way, it’s obvious that we’re just filtering the collection based on a condition. So, just do that.

var hazardNear = false
        
if hazardCount > 0
{
    hazardNear = exits[room].filter{hazardRooms.contains($0)}.count != 0
}
        
return hazardNear

We’re filtering the array of rooms for ones that contain the hazard and checking for a non-zero count. But this is still a bit clunky. Let’s examine what we’re really asking.

var hazardNear = false
        
if hazardCount > 0
{
    hazardNear = !exits[room].filter{hazardRooms.contains($0)}.isEmpty
}
        
return hazardNear

It’s far more clear to simply ask if the filtered array is non-empty directly. What’s left is one final simplification.

return hazardCount > 0 && !exits[room].filter{hazardRooms.contains($0)}.isEmpty

One might ask whether having an explicit count is necessary as the hazard array has a count. True. Although the hazard array is built based on the number of hazards, so I still need it to be around.

Overall, I fairly pleased with the results of the exercise. In the end, the code is much clearer in intent.

Read Full Post »

Failure Is Not An Option, It’s A Requirement

Posted in Books, Software Development, tagged , on 11 Jun 2016| Leave a Comment »

One of the nice things about taking months or even years to finish reading a book is that it gives me plenty of time to reflect on it. Sure, I could binge read, but the material is far less sticky. The longer I spend with a book, the more connections I can make. “Failure Is Not An Option,” which I just completed a few days ago, is a good example of this.

The book itself is a retelling of history of the manned space program from the point of view of someone intimately familiar with the events. At first blush, it would appear to have nothing to do with the process of software development. The Apple Pencil I’m using at the moment probably has more computing power than anything available at the time. So, what is the connection?

It is absolutely true that when dealing with matters of human life that failure is not an option. When you look at the history of American manned space flight from the outside, the only thing you see is the ultimate expression of this. You see successful execution, even in the case of unforeseen situations. And that’s the key, it’s from the outside. Success was not achieved because perfect people perfectly executed perfect plans using perfect equipment under perfect conditions. Yes, there were times when the execution, equipment or conditions were perfect. Most of the time, success was ensured inspite of less than perfect conditions. For me that’s the story.

During the space program, failure was a mandate. When you exist in a world of constrained resources, the best way to ensure that you will be successful is to see how you respond when the resources that you depend upon are available. But, wouldn’t you always being everything you needed and have it on hand? One would hope so, but let’s say, for the sake of argument, that you didn’t. What then? If something bad happens, how much time do you have to get things back on track before it’s game over? Well, that depends on what happened, when it happened and how important it was.

Okay, sure, fine, but what did they do and what does it have to do with software development?

In the case of the space program, failure was ensured before the mission. Every possible they could conceive of. Teams of people had the sole job of enducing failures during mission simulations. We’re not talking SimCity or Second Life here. These simulations were conducted in physical hardware as identical as possible. The difference is that this user interface was driven not by physical sensors, but computers. Crews were made to experience failure over and over until their responses were second nature. When you think about it, this is no different from any physical endeavor. The more you train yourself to respond to situational changes, the better your performance becomes.

As of late, I believe that many of the problems with software and hardware have been coming about because of a focus merely on the “happy path.” The idea that we should worry first about making it work and then later about failure conditions. Unfortunately, once management sees something that “works,” they  are unlikely to allocate time to break things. The situation has been made worse by the attitude that software will be tested in beta.

As a result, not only is inferior quality software being produced, but the software is being used in environments which expose their data to exfiltration. We see this is malware infecting point-of-sale systems. We see it in OpenSSL and other open source code. Software being widely adopted under the assumption that someone else must be testing it. Entire generations argue for speed over safety.

So, why has this attitude taken hold? Is it harder to design for failure first? Not really. In fact, it makes unit testing easier as it builds in the failure paths before the actual implementation. But doesn’t this make the code slower? Not implicitly. Between the use of modern tool chains and profile guided optimization, the cost of fail first, fail fast is minimal. Why don’t we do privilege segregation? For the same reason we don’t apply principles of  MVC, MVVM or VIPER. People sit in front of a screen and type. They use the excuse of doing “agile” development on short cycles for not properly planning. Now, I have nothing against short cycles, but to be used properly, you have to accept that not every problem can be evaluated, solved, implemented and tested in 2 or 3 weeks. Some things are hard. Some things can only be accomplished by specific individuals. Some tasks have dependencies on outside resources. Software isn’t building IKEA furniture. There are high level of indeterminacy that can and do crop up.

Neither can we pretend that the software we create lives in isolation or that security is the responsibility of the user. Holding onto data simply because it would make the developer’s job easier during development is not enough of a reason. Not using OS provided encrypted data storage because it makes it harder to debug is simply lame. Transmitting data in the clear or without access control just invites both data exfiltration and command-and-control injection.

Here’s the thing. If you want to succeed, you must fail. You must fail first and you must fail fast. Failure helps to characterize the system. It helps in the creation of documentation. It helps validate the design. Failure makes gives you a better understanding of the problem space. In short, failure is not an option; it is a requirement.

Read Full Post »

Unrecognizable – Swift 3.0 Has No Interface Builder Love

Posted in Apple, Software Development, tagged , on 26 May 2016| Leave a Comment »

One of the things I’ve always liked about Apple is the way they revisit and revise their tools. Although there is comfort in logging on to any *nix system and knowing that ed or vi or gcc will be there, I believe that the lack of investment in standard tools hurts more than knowing that anyone who has written code since 1980 will still be at home. This is the technical equivalent of having a rotary gang switch to change channels on your flat-screen TV because that’s what your grandparents were used to and they still need to use the TV. You know what, they managed to adapt to there being a wireless remote and 500 channels.

Swift is probably the most visible of today’s Apple tool changes. As we fast approach the release of Swift 3.0, it’s good to check in and see what the future of software development of Apple products is going to be like. This is one of those growth and pain stories.

From the language standpoint, Swift 3.0 brings a greater level of consistency in terms of behavior and style. Compared to other languages, new developers will have fewer exceptions to keep track of. For those following the latest Stanford iOS class on iTunesU (CS193P) thinking that when they’re done that they’ve learned Swift, well, sorry, but in a few months you’ll need to update all your code.

One of the biggest changes in Swift 3.0 is the way that the Apple OS-specific libraries are handled. In the same way that C++ and Python aren’t bound to a particular operating system, neither is Swift. You don’t need to be on MacOS or iOS to use Swift. The problem is that most applications that interact with people need to be bound to some kind of operating environment. On Windows that’s typically done by using .Net. With Java, you’ve got a laundry list of environments (EE, SE, ME). For Swift, you’re binding to Apple’s plenitude of support libraries. All of which have, for the past decade or so, been based on Objective-C. The great thing about new languages is that they incorporate lessons learned. The devil of them is that vast body of existing code they need to interface to.

In the case of MacOS and iOS, the libraries make a lot of sense if you’re writing code in 2004. We’ve learned a lot about languages and frameworks and operating systems since then. But, as with all things, there are only so many hours in the day. Inevitably, you build a core around the language and, if you’re nice, create bridges (shims) to the external bits. If you’re Apple, you’re usually particularly nice and create toll-free bridges (ones built for you). If you’re Microsoft, you have the joy of multiple layers of conversions to look forward to. Eventually as the language evolves, you get to the point where the rules you gave people as to how the bridging worked do align with the language. That’s where we are today with Swift 3.0.

So, by now, you’re saying to yourself that this must be really bad to merit such a long preamble. Sort of. I came across a nifty article on what’s new in Swift 3.0 and decided to download the May 9th Swift trunk [see update below] and see for myself. There’s a nifty video from the same person. He does a good job of covering the changes. Remember my earlier reference to the Stanford class? I’ve been following it myself. I thought, “let’s see how what’s being taught tracks.” For my first attempt I chose a small app that uses gestures to tinker with a face drawn to the screen. Now as those of you familiar with iOS will be aware, there are two ways of implementing UI binding to code. You can either write it in code or use Interface Builder. The thing about using IB is that it handles a bunch of tedious details for you. It also hides stuff and is the source of weird errors if you screw things up.

In implementing this app, I of course used IB. And you know what, I crashed and burned. After installing the new tool chain and restarting Xcode, my modest app showed a variety of error stemming from the changes in Swift 3.0, all of which were easily corrected thanks to the very clear error messages and suggested remediation provided. The app launched and drew the baseline image, but any gesture failed miserably. Maybe it was the project. I created a new project with nothing but a raw view and added a tap recognizer in IB and wired it up to the view with a print diagnostic. It happily built, but also crashed on tap.

2016-05-26 15:16:31.150 f2[21677:736529] -[f2.ViewController tap:]:
  unrecognized selector sent to instance 0x7ffe59d31eb0
2016-05-26 15:16:31.156 f2[21677:736529] *** Terminating app due to
  uncaught exception 'NSInvalidArgumentException', reason:
  '-[f2.ViewController tap:]: unrecognized selector sent to instance
  0x7ffe59d31eb0'

Creating the action in code worked. To be fair, this tool chain is coming from the Swift site and not Apple. I then checked to see what happened if I tossed in a vanilla button. Same problem. That wouldn’t do. The answer turned out to be held in the AppDelegate. As it happens the new tool chain generated a warning for each of the standard boiler functions.

/Users/charleswilson/Documents/projects/2016_CS193P/FaceIt/FaceIt/
  AppDelegate.swift:17:10: Instance method
  'application(application:didFinishLaunchingWithOptions:)' nearly matches
  optional requirement 'application(_:didFinishLaunchingWithOptions:)' of
  protocol 'UIApplicationDelegate'

The provided code reads:

func application(_ application: UIApplication,
  didFinishLaunchingWithOptions launchOptions: [NSObject: AnyObject]?)
  -> Bool

It seems that the functions need additional decoration. The warning notes that the function nearly matches. Allowing FixIt to do its thing yeilded:

@objc(application:didFinishLaunchingWithOptions:)
  func application(_ application: UIApplication,
  didFinishLaunchingWithOptions launchOptions: [NSObject: AnyObject]?)
  -> Bool

Apparently, additional decoration is required to bridge properly. So I went back to the standard IB added recognizer. Here’s the action.

@IBAction private func tap(sender: UITapGestureRecognizer)

Once the objc decoration is added, all is well.

@IBAction @objc(tap:) private func tap(sender: UITapGestureRecognizer)

Not a big deal to make the correction once you realize what’s required. Hopefully, others will take this as a heads up and not be bogged down as they update their code.

Overall, I have a high degree of confidence in Swift 3.0, the changes make the language better. So, go get it yourself and see the future of Apple software development.

2016-06-05 Update

I pulled the May 31 Swift trunk with the same results. I’d imagine that Apple will clean this up really soon as WWDC is just around the corner.

2016-06-09 Update

I pulled the June 6 Swift trunk and got the same results. Cutting it close for WWDC.

Read Full Post »

-Wall, Compiler Warnings and Other Lies Software Developers Believe

Posted in Computers, Software Development, tagged , , , , on 17 May 2016| Leave a Comment »

An interesting thing about having spent over thirty years taking software from one platform to another is that, time and again, I’ve had my understanding of what constitutes correct code challenged. That’s a good thing.

Sadly, many people who ply the trade of software development mistakenly believe that a compiler has the ability to warn you when you’re code is going to behave in ill-advised ways. Worse yet, they fall into the trap of believing either that their code is correct if it compiles without warnings or that if the compiler accepts your code then any compiler will. Unfortunately, these beliefs are the developer equivalent of a two year old’s lack of object permanence. These two tragedies aside, the vast majority of developers are clueless as to how static analysis can and should be used to ensure code quality.

Let’s rewind a bit and work through these.

In the beginning was the language specification. It was a bright, shiny idea given form. Lest you get the idea that these documents, venerated by both compiler authors and language wonks alike, are intrinsically sane; please recall that the original Ada spec allowed minus signs in the middle of numeric literals and that 8 and 9 were perfectly acceptable octal digits in C. Now a computer language without a compiler is fairly dull. Enter the compiler authors. These individuals, who number about 1000 in the world and of whom I’ve personally known about a dozen, are highly proficient at taking the language specification and giving it life. The way they do this is far more Pollock than Vermeer. Why? Well, a language is the embodiment of a worldview. Unlike source code control systems, which are only created when someone gets fed up with the way that the current one they’re using does one particular thing to such an extent that they can’t bear living under its yoke any longer, languages come from the world of paradigms. Why? If your pet peeve is small, you’ll probably be able to either work around it or get it added to the language (typically contingent upon who you know). If your peeve isn’t small, any attempt to modify the language you’re using will have the same result as updating the value of Planck’s constant or dropping a storm trooper platoon into the middle of a city on Vulcan.

It can’t possibly be that bad you say. Actually, it can. And, in fact, it is. A long time ago, Apple was transitioning from it’s Pascal-based OS to a C-based one. The transition was fraught with byte-prefixed, null-terminated strings. The resultant code was pretty horrific. Just recently, I was working on a feature that required me to move between BSTR, wstring, COM and char strings. This because the language wasn’t designed with the notion that strings could be more than just US English. Compare this to Swift which is a pure Unicode language right down to the variable names.

Every compiler writer brings their own unique experiences and skill set to realizing the worldview embodied in the language specification. No two compilers will realize it the same way. Oh, they’ll be close and probably agree 90% of the time. The biggest area of difference will be in what each compiler considers important enough to warn the developer about. On one end of the scale, you could argue that if the language specification allows something that the developer is free to write the code accordingly. At the other, the compiler would report every questionable construct and usage. All compilers that I’m aware of fall somewhere in the middle.

Unfortunately, rather than biting the bullet and forcing developers to recognize their questionable and problematic coding choices, compilers have traditionally allowed code to have a pass by not enforcing warnings as errors and by having the default warning level be something uselessly low. To make matters worse, if you do choose to crank up the warning level to its highest severity, many times, the operating system’s headers will fail to compile. I’ve discovered several missing macro symbols that the compiler just defaulted to 0. Not the expected behavior. Microsoft’s compiler have the sense to aggregate warnings into levels of increasing severity. gcc however, does not. The omnibus -Wall turns out to not actually include every warning. Even -Wextra leaves stuff out. The real fun begins when the code is expected to be compiled on different operating systems. This can be a problem for developers who have only ever worked with one tool chain.

So, let’s say that you realize that compilers on different platforms will focus on different issues. You’ll probably start considering trying other compilers on the same platforms. Once all those different compilers, set to their most severe, pass your code is good right? Not so fast. Remember, a compiler’s job is to translate the code, not analyze it. But, you do peer reviews. Tell me, how much of the code do you look at? How long do you look at it? Do you track all the variable lifetimes? Locks and unlocks? In all the code paths? Across compilation units?

Of course you don’t. That would be impossible. Impossible for a person. That’s why there are static analysis tools. My current favorite is Coverity. And yes, it costs money. If you can sell your software, you can pay for your tools.

But the fun doesn’t end there. Modern compilers can emit additional code to allow for profile guided optimization (PGO). Simply put, instrument the code, run the code, feed the results back into the compiler. Why? Because hand tweaking the branch predictors is an exercise in futility. Additionally, you’ll learn where the code is spending its time. And this matters because? It matters because you can waste a lot of time guessing where you need to optimize.

Finally, there’s dynamic analysis. This is realm of run-time leak detection. Wading through crash dumps and log files is a terrible way to spend your time.

So, are you developing quality software or just hacking?

Read Full Post »

Shelfware Keeps Me Learning

Posted in Computers, Software Development, tagged , , on 08 May 2016| Leave a Comment »

There’s an old computer joke. “There are 10 kinds of people. Those who count in binary and those who don’t.”

I think there are two kinds of software developers, those who constantly reexamine their skill set and those who like working on antique cars. Mind you, I have nothing against antique cars. A good chunk of my personal library is dedicated to computer history. I love reading about how we got to where we are. I’ve been privileged to have been an observer to some of that history. That being said, ours is a discipline where answers are only as good as the half-life of their questions.

What’s the lifetime of a question you ask. It depends. If the question is “What is the length of the diagonal of a right triangle given the lengths of the other two sides?”, pretty damn long. On the other hand, if it’s “What is the best way to teach people how to write a program?”, well, tick-tock.

So, what does this have to do with shelfware? As it happens, a fair number of the projects I’ve worked on have become shelfware. Sometimes it was tech that the world just wasn’t ready for. Other times, it was a case of the tank shell and mosquito. Way too big of a hammer for the problem. (The bigger the hammer, the greater the resources needed to create and eventually wield it.)  There are those time when you’ve created a solution either without a problem or by the time you’ve got your solution, the problem has either been solved another way thank you very much, or simply no longer exists. (Remember how everyone got all wound up about Y2K?) Finally, some projects simply run out of time.

The interesting thing is that no one has as their goal the creation of shelfware. But, you know what? Shelfware happens. So, what is one to do? My favorite exercise in such is the post mortem. As with all things, I ask my two questions. “What do I have?” and “What do I need?” What I have is easy to answer. It’s one of the above cases. What I need is to determine is how it got there and what can I do to prevent the same sequence of events from occurring again.

Part of what comes from this analysis help me improve my situational awareness. Contrary  to popular belief, it is not enough to be good at what you do and expect that your insight, code or experience will suffice. Software is like an elaborate dining experience. If you can’t get the right dishes out to the right people in the right order at the right time, there will be hell to pay. By the same token, if your team isn’t able to fire on all cylinders, you’re bound to either lose the race or crash and burn. Eventually. Nothing is more pathetic than a software team who’s sole purpose is to sustain a dead product. But I’ll leave zombieware for another day.

Situational awareness is all well and good, but it’s a external. The more important thing for me is to see what I can do to up my personal game. This is not to say that I wait for a train wreck to do this, only that unlike my usual mental fitness endeavors (consuming all the Apple WWDC and other conference sessions, Coursera classes, etc.) and scanning the horizon for interesting directions that the industry is moving toward, I like to take stock of my core skill set.

It takes skill to drive a nail with one or two hammer strikes, but it’s far more efficient to use a pneumatic nail gun. More fun too. And you don’t need the same level of skill. The same is true of software development. In the early days of both C and C++, there were no libraries. Everything was handcrafted in every single program. The number of times any given problem was solved, to varying degrees of success and quality, is staggering. People created problem-specific solutions which were then expanded upon for other projects. As usually occurs, they accreted unrelated and inseparable bits. By the time an opportunity to refactor the code base into some semblance of modernity, it is usually deemed too great a risk. So, while you’re becoming the expert in dinosaur care and feeding, the next generation of developers has been tinkering away with the latest and greatest.

I know, I know, the fundamentals you say. Okay. So, how about you show my your linear and logarithmic interpolation skills? No? You used Matlib perhaps? Hmm. Well then, how about explaining why 8.3 file names are storage efficient? Surely balancing binary trees. But wait, don’t you get that from maps in C++ and just about every other modern language? The finer details of memory allocation and pointers? Have you ever talked to a Java or Python developer? Binary tree common ancestors? Understanding how to implement that must be important. Seriously? In thirty-two years of developing software full time, I have never needed to re-balance a binary tree or find a common ancestor, except for interview questions. Which probably makes me unqualified to create the next great search algorithm. And that’s okay. All these answers reached their half-life long ago. Well, maybe not Tango trees. But they’re still a solution looking for a practical problem.

Over time, all these problems have had their solutions codified into either the language or the language’s standard library. This is a good thing. Map / reduce was once new and shiny. Now it’s built into Apple’s Swift language. I would much rather people who live to create the best sorting algorithm do so. That way everyone benefits. I don’t want to have to care if my data is partially sorted, whether I have fifty or fifty million items, if I have strings or structures. I want to sort my data in order to do something else with it, not because sorting it amuses me. By the same token, no one in their right mind would even consider creating a GUI from the ground up today. I’ve personally had to do this more than once. It’s a boat load of work and not portable. Why should data structures and their manipulation be any different? After all, we create abstractions so that we can handle problems of greater complexity.

So, if I’m not looking at these, then what?

Currently, three things have my attention. The first is Apple’s Swift. I’ve been watching this language develop over the past two years into a nearly complete solution for application creation. The second is C++11/4/7. I started working with C++ in the mid ’90s and it has managed to keep pace with developments in the processor and compiler spaces. Anyone who is still writing code as if nothing has changed since C++03 really needs to look at C++14. Between auto, atomics and memory handling changes, C++ continues to bring its A game. Finally, and most importantly, I’ve been looking at secure coding. Having recently taken the US CERT secure coding course, I was surprised at the level of code exploitation possible because of patterns of coding that go back decades. If OpenSSL has issues being exposed on a near weekly basis, would we expect that other major pieces of our infrastructure are not equally at risk?

To remain relevant in the world of software development, one must not only be competent in the current use of technology, but also be actively learning and incorporating the advances in the materials (languages / frameworks), methodologies (paradigms) and tools (editors, compilers, IDEs, etc.). Not to mention other peoples’ code.

Everything can and should be learned from,  even shelfware. It’s merely experience without exposure.

Read Full Post »

Seeking Insecurity: Taking CERT’s C/C++ Security Class

Posted in Computers, Education, security, Software Development, tagged , , , , , , , on 26 Mar 2016| Leave a Comment »

Apparently, I am the first person to complete SEI CERT’s online version of their course in C/C++ secure coding. With all my pausing the videos to take notes, it took me forty hours to get through, but it was worth the time. Actually, it’s two courses, one in security concepts and the other in secure C/C++ coding.

There’s four full days worth of material presented. The videos are chunked into 2 – 60 minute segments, which is helpful as the presenters are speaking to an actual class. This choice of format is both good and bad. Good in that you see real questions being asked. Bad in that, as with many classroom situations, the presenters go off into the weeds at times.

There are six exercises covering major elements (strings, integers, files, I/O, etc.). For these, web interfaced Debian Linux VMware VMs are provided. These boot quickly and have all tools required to perform the exercises. An hour is allotted for each, which is plenty of time. Following each is a section covering the solutions. This is especially helpful as there’s no one to ask questions of during. It’s also possible to download a copy of the VM being used. Sadly, no provision for Windows or native Macintosh environments.

Two books are provided (in various formats). The Secure Coding in C and C++, Second Edition and The CERT® C Coding Standard, Second Edition: 98 Rules for Developing Safe, Reliable, and Secure Systems (2nd Edition).

Generally, I found the material was well delivered. The content in the areas of (narrow) strings, integers and memory handling were exhaustive. The sections on file handling and concurrency were almost completely lacking any Windows coverage. Given that number of systems used by businesses and individuals based on Windows, this is quite disappointing. There was also a bit of hand waving around the use of wide characters. Not providing complete coverage here is a true deficiency as properly dealing with the reality of mixed string types is a reality that isn’t going to go away.

As to this being a C and C++ class, well, sort of. There are nods to C++, but like so much code out there, the C++ is tacked on the side. It really should be treated as a separate class as is done with Java. Many recommended mitigations can with the proviso that they were not portable. I think that when you’re dealing with security, the fact that there exists a mitigation outweighs its portability. These things can be abstracted to achieve portability. As someone who’s spent a fair amount of time doing cross-platform development (Unix-MacOS, MacOS-Windows, Linux-Windows), these are important issues to me.

To those who argue that securing code will make it slow, I would ask what a single security compromise would cost their company in reputation and direct monetary terms.

Would I recommend the class? Yes. There are important concepts and real-world examples on display here. I challenge anyone to take this class and not be horrified by the way most C code is written.

Could I teach this material? Definitely.

Read Full Post »

HackerRank Using Apple’s Swift

Posted in Apple, Computers, Software Development, tagged , , , , on 06 Jan 2016| Leave a Comment »

HackerRank has become a popular showcase for software developers. Its many participants have the ability to choose their language. Recently Apple’s Swift language has been added.

This is all well and wonderful, but as I written about in previous posts, non-UI oriented use of Swift has not been a priority. It’s not impossible, but you have to scrounge for the bits you need to use it.

As those who have used HackerRank know, non-UI is the way of things. So, here are a few bits that make it possible to use Swift in the HackerRank.

Note: In production code, one should always check when unwrapping variables. In HackerRank, input will be well-behaved and the focus is on the internal workings rather than the input mechanism.

Reading a string from the input stream

let string_value = readLine(stripNewline: true)!

Reading a single number from the input stream

let numeric_value = Int(readLine(stripNewline: true)!)!

Reading a string array from the input stream

let stringArray = input.characters
                       .split {$0 == " "}
                       .map (String.init)

Reading a numeric array from the input stream

let numbericArray = input.characters
                         .split {$0 == " "}
                         .map (String.init)
                         .map {(var s) -> Int in return Int(s)!}

Reading mixed values from the input stream

let stringArray = input.characters
                       .split {$0 == " "}
                       .map (String.init)

let iValue1 = Int(stringArray[0])!
let sValue  = stringArray[1]
let iValue2 = Int(stringArray[2])!

Writing single values to the output stream

print(foo)

Writing values in string to the output stream

print ("There are \(found) or \(expected) things")

Writing array values to the output stream

print (array.map { String($0) }.joinWithSeparator(" "))

Why not use extensions, you may ask. You can do that if you like. I thought it would be better here to show the minimal elements. And generally HackerRank in very limited in its input and output aspects.

This should cover all the usual cases for input and output to enable you to use Swift with HackerRank. So, the next time you’re using HackerRank and you don’t see Swift as an option, be sure to ask the author to add it.

Read Full Post »

Swift Command Line Input: Truth in Code

Posted in Apple, Computers, Software Development, tagged , , on 21 Dec 2015| 2 Comments »

This is my third post on the Swift command line. In the first, I wrote on how to do command line input with Swift. In the second, I cleaned up the code after a year of tinkering with Swift. In this third installment, I get rid of more Objective-C.

Almost a month ago, Apple released Swift into the open source community. Pretty spiffy that. So, being the language wonk that I am, I bopped over to the Swift site and downloaded the sources. After a bit of scrounging, I discovered that there existed a standard library call to read a line from the input stream. I was a bit disappointed that I’d not seen this talked about anywhere, but that’s life.

So, I’ve updated my code sample and present it below. I do some forced unwrapping as the point is to highlight the process. Be sure to code defensively.

I’ll probably come back to this one more time once I’ve had a chance to look more into the Swift standard library so I can eliminate my untidy Objective-C dependent putString() routine.

//
//  swift_intput_routines.swift
//
//  Created by Charles Wilson on 9/27/2014.
//  Revised by Charles Wilson on 9/27/2015.
//  Revised by Charles Wilson on 12/21/2015.
//
//  Copyright (c) 2014 – 2015 Charles Wilson. All rights reserved.
//
//  Permission is granted to use and modify so long as attribution is made.
//

import Foundation

func putString (outputString : String = “”)
{
if !outputString.isEmpty
{
NSFileHandle.fileHandleWithStandardOutput().writeData((outputString as NSString).dataUsingEncoding(NSUTF8StringEncoding)!)
}
}

func getString (prompt : String = “”) -> String?
{
if !prompt.isEmpty
{
putString(prompt)
}

return readLine(stripNewline: true)
}

func getCharacter () -> Character
{
let inputValue     : UInt32      = UInt32(getchar())
var inputCharacter : Character

inputCharacter = Character(UnicodeScalar(inputValue))

return inputCharacter
}

func getInteger (prompt : String = “”) -> Int
{
if !prompt.isEmpty
{
putString(prompt)
}

return (getString()! as NSString).integerValue
}

func getFloat (prompt : String = “”) -> Float
{
if !prompt.isEmpty
{
putString(prompt)
}

return (getString()! as NSString).floatValue
}

Here’s the main to exercise it.

//
//  main.swift
//
//  Created by Charles Wilson on 9/27/2014.
//  Revised by Charles Wilson on 9/27/2015.
//  Revised by Charles Wilson on 12/21/2015.
//
//  Copyright (c) 2014 - 2015 Charles Wilson. All rights reserved.
//
//  Permission is granted to use and modify so long as attribution is made.
//

import Foundation

var name = getString("What is your name? ")!

if name.isEmpty
{
    name = "George"
    
    putString("That's not much of a name. I'll call you '\(name)'\n")
}
else
{
    putString("Your name is '\(name)'\n")
}

let age = getInteger("How old are you \(name)? ")

putString("You are \(age) years old\n")

// get a value without a prompt
let number = getInteger()

putString("\(number) is a nice number\n")

let floating = getFloat("Enter a floating point number: ")

putString("\(floating) works for me\n")

var c : Character

putString("Type stuff. Enter ^ when done.\n")

let sentinel : Character = Character(UnicodeScalar("^"))
var in_c     : Character

repeat
{
    in_c = getCharacter()
}
while ( in_c != sentinel )

putString("\n\n")
putString("bye\n")

Read Full Post »

Notebooks By The Numbers: My Work in 2015

Posted in Computers, Software Development, tagged , , , on 20 Dec 2015| Leave a Comment »

I’m a big fan of notebooks. Whoever has the unenviable task of sorting through my stuff after I’ve made that final blog entry will be bored to tears with the piles of notebooks they’ll have to go through, assuming that they don’t simply rent a dumpster and chuck the whole pile. Not my problem though.

To be fair, the consistent use of notebooks to record a project’s designs, progress and failures does not make you a da Vinci. The world is replete with pen wielding lunatics. That being said, I believe that using notebooks (actual paper ones) helps give shape to nascent ideas. The simple act of putting pen (or super high-tech stylus) to paper (be it college rule loose leaf or archival quality) forces you to think a bit more about what you’re doing. Perhaps this is why I’ve never been much of a fan of those who believe that they can create a complex piece of software without some form of design. I’m not talking space shuttle here. A simple data flow diagram, threat model and use case statement is all I ask (the basics). These are the bare minimum.

Whiteboards a all well and fine, but unless you’ve got a dedicated scribe hanging about, what happens on the whiteboard tends to stay on the whiteboard. Besides as a clever person once said, “I couldn’t reduce it to the freshman level. That means we don’t really understand it.”

At work, my notebooks are a mix of the mundane and the technical. I track meetings, conversations, designs and random thoughts. I find this helpful when trying to make sense of all the balls in the air and occasional ones that end up on the ground. At the end of each year, I participate in the great year-end summary exercise. For this my notebooks are invaluable. This year was no exception. What was exceptional was the numbers.

  • 193 code reviews (other people’s)
  • 81 user stories / bugs (evaluation, implementation, review)
  • 24 technical documents (papers, presentations and the like)

Once I’d finished doing my sums, I went back to my 2014 summary. Not even close. I’m not really sure why.

What did I do in 2015?

I spent a lot of time in some very specific areas:

  • writing security code
  • developing threat models
  • teaching Visual Studio to Linux developers
  • improving code quality through use of static analysis and compilers
  • coordinating open source and third-party library use

What did I learn in 2015?

  • developers expect a language to work based on the platform they learned on and not the spec (which no one ever reads)
  • people really, really don’t understand the difference between authentication and authorization
  • developers don’t see security as their problem
  • developers don’t trust static analysis (The code has never failed, so why is that change important?)
  • talking about cyclomatic complexity metrics is like trying to explain Klingon opera to a Star Wars fan

What’s up for 2016?

Who can say. Security doesn’t look like a bad guess. And it really doesn’t matter how fast machines get or how much storage they have. Programs will grow to consume all available resources.

The only certainty is that I’ll be filling another set of notebooks.

Read Full Post »

« Newer Posts - Older Posts »