Thread-Safe Random Access to Zip Archives

Many games choose to store their resources in packages instead of shipping the potentially thousands of individual files directly. This is sometimes an attempt at tamper-proofing, but mostly it is about performance. Try copying a thousand 1 KiB files from one drive to another, then copy a single 1 MiB file on the same way – the former operation will take many times longer.

A good choice for a package format is the well known .zip archive. It’s not necessarily well-designed, but it decompresses fast and you definitely won’t have any problems finding tools to create, modify and extract .zip archives. Thus, when I started work on a file system abstraction layer for my future games, seamless .zip support was one of my main goals (I may also add 7-Zip at a later time just for fun).

Here is the design I came up with after exploring the file system APIs available on Windows, WinRT and Linux:

UML diagram showing the design of my file system abstraction layer

You may notice some rather radical design choices in my File class: there are no Open() or Close() methods and there is no Seek() method, either – each read or write specifies the absolute position without requiring the file to be opened.

Read More

Ogre 1.8.0 for WinRT/Metro

Ogre 3D Logo

In March I provided some binaries of Ogre 1.8.0 RC1 that were based on Eugene’s Metro port of Ogre, allowing Ogre to run as a native Metro App, using the Direct3D 11 renderer and RTShaderSystem for dynamic shader generation.

Those binaries no longer work with the Windows 8 Release Preview and Visual Studio 2012 RC, so I thought I’d provide an updated package!

Screenshot of Ogre 1.8.0 on Windows 8 Release Preview running as a Metro app

This time I went a bit further: while the last package was compiled with multithreading disabled, I have in the meantime ported Boost 1.50.0 to compile on WinRT (using a slightly modified version of Shawn Hargreaves’ WinRT CreateThread emulation code. Thus, this Ogre build has full support for multithreading and includes Boost!

Read More

Code Better: Headers without Hidden Dependencies

When you work on a larger project, you cannot easily keep track of which header depends on which other header. You can (and should) do your best to keep the number of other headers referenced inside your headers low (to speed up compilation) and move as many header dependencies as you can into your source files, but this still doesn’t prevent you from building headers that implicitly depend on another header being included before them.

Take this example:

#ifndef MAP_H
#define MAP_H

/// <summary>Stores the tiles of a 2D map<summary>
struct Map {};

#endif // MAP_H
#ifndef WORLD_H
#define WORLD_H

#include "Actor.h"
#include <vector>
// Oops, forgot Map.h, but won't notice since World.cpp includes Map.h before World.h

/// <summary>Maintains the state of the entire world<summary>
struct World {
  /// <summary>Stores the map as a grid of 2D tiles</summary>
  public: Map Map;
  /// <summary>Actors (player, monsters, etc.) currently active in the world</summary>
  public: std::vector<Actor *> Actors;
};

#endif // WORLD_H

Throughout your project, map.h might always end up being included before world.h and you might never notice that if someone included world.h on its own, a nasty compilation error would be the result.

So what can you do ensure this situation never happens?

Read More

How to Delete Directories Recursively with Win32

Well, while I’m at it, here’s the counterpart to the recursive directory creation function from my last post, a function that recursively deletes a directory and all its contents.

Ordinarily, you could just use the shell API to achieve this on classic Win32:

/// <summary>Deletes a directory and everything in it</summary>
/// <param name="path">Path of the directory that will be deleted</param>
void deleteDirectory(const std::wstring &path) {
  std::vector doubleNullTerminatedPath;
  std::copy(path.begin(), path.end(), std::back_inserter(doubleNullTerminatedPath));
  doubleNullTerminatedPath.push_back(L'\0');
  doubleNullTerminatedPath.push_back(L'\0');

  SHFILEOPSTRUCTW fileOperation;
  fileOperation.wFunc = FO_DELETE;
  fileOperation.pFrom = &doubleNullTerminatedPath[0];
  fileOperation.fFlags = FOF_NO_UI | FOF_NOCONFIRMATION;

  int result = ::SHFileOperationW(&fileOperation);
  if(result != 0) {
    throw std::runtime_error("Could not delete directory");
  }
}

But WinRT/Metro applications cannot use the shell API and have to do it manually. So here’s a piece of code that takes care of directory deletion using nothing but Win32 API calls that are also available to WinRT/Metro applications:

/// <summary>Automatically closes a search handle upon destruction</summary>
class SearchHandleScope {

  /// <summary>Initializes a new search handle closer</summary>
  /// <param name="searchHandle">Search handle that will be closed on destruction</param>
  public: SearchHandleScope(HANDLE searchHandle) :
    searchHandle(searchHandle) {}

  /// <summary>Closes the search handle</summary>
  public: ~SearchHandleScope() {
    ::FindClose(this->searchHandle);
  }

  /// <summary>Search handle that will be closed when the instance is destroyed</summary>
  private: HANDLE searchHandle;

};

/// <summary>Recursively deletes the specified directory and all its contents</summary>
/// <param name="path">Absolute path of the directory that will be deleted</param>
/// <remarks>
///   The path must not be terminated with a path separator.
/// </remarks>
void recursiveDeleteDirectory(const std::wstring &path) {
  static const std::wstring allFilesMask(L"\\*");

  WIN32_FIND_DATAW findData;

  // We'll iterate over the direct contents of the directory in any order
  std::wstring searchMask = path + allFilesMask;
  HANDLE searchHandle = ::FindFirstFileExW(
    searchMask.c_str(), FindExInfoBasic, &findData, FindExSearchNameMatch, nullptr, 0
  );
  if(searchHandle == INVALID_HANDLE_VALUE) {
    DWORD lastError = ::GetLastError();
    if(lastError != ERROR_FILE_NOT_FOUND) { // or ERROR_NO_MORE_FILES, ERROR_NOT_FOUND?
      throw std::runtime_error("Could not start directory enumeration");
    }
  }

  // Did this directory have any contents? If so, delete them
  if(searchHandle != INVALID_HANDLE_VALUE) {
    SearchHandleScope scope(searchHandle);
    for(;;) {

      // Do not process the obligatory '.' and '..' directories
      if(findData.cFileName[0] != '.') {
        bool isDirectory = 
          ((findData.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) != 0) ||
          ((findData.dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) != 0);

        // Subdirectories need to be handled by deleting their contents first
        std::wstring filePath = path + L'\\' + findData.cFileName;
        if(isDirectory) {
          recursiveDeleteDirectory(filePath);
        } else {
          BOOL result = ::DeleteFileW(filePath.c_str());
          if(result == FALSE) {
            throw std::runtime_error("Could not delete file");
          }
        }
      }

      // Advance to the next entry in the directory
      BOOL result = ::FindNextFileW(searchHandle, &findData);
      if(result == FALSE) {
        DWORD lastError = ::GetLastError();
        if(lastError != ERROR_NO_MORE_FILES) {
          throw std::runtime_error("Error enumerating directory");
        }
        break; // All directory contents enumerated and deleted
      }

    } // for
  }

  // The directory is empty, we can now safely remove it
  BOOL result = ::RemoveDirectoryW(path.c_str());
  if(result == FALSE) {
    throw std::runtime_error("Could not remove directory");
  }
}

This code is free for the taking and you can use it however you want.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

How to Create Directories Recursively with Win32

As I just found out, the CreateDirectory function on Win32 can only create one directory at a time. If one, for example, specifies C:\Users\All Users\FirstNew\SecondNew as the directory to create, and both FirstNew and SecondNew do not exist, then CreateDirectory() fails.

That’s less than ideal for some cases. Recently, for example, I wanted my game to create the C:\Users\<Whoever>\Documents\My Games\<GameName> directory, where both My Games and <GameName> may not yet exist. Here’s a workaround:

/// <summary>Creates all directories down to the specified path</summary>
/// <param name="directory">Directory that will be created recursively</param>
/// <remarks>
///   The provided directory must not be terminated with a path separator.
/// </remarks>
void createDirectoryRecursively(const std::wstring &directory) {
  static const std::wstring separators(L"\\/");

  // If the specified directory name doesn't exist, do our thing
  DWORD fileAttributes = ::GetFileAttributesW(directory.c_str());
  if(fileAttributes == INVALID_FILE_ATTRIBUTES) {

    // Recursively do it all again for the parent directory, if any
    std::size_t slashIndex = directory.find_last_of(separators);
    if(slashIndex != std::wstring::npos) {
      createDirectoryRecursively(directory.substr(0, slashIndex));
    }

    // Create the last directory on the path (the recursive calls will have taken
    // care of the parent directories by now)
    BOOL result = ::CreateDirectoryW(directory.c_str(), nullptr);
    if(result == FALSE) {
      throw std::runtime_error("Could not create directory");
    }

  } else { // Specified directory name already exists as a file or directory

    bool isDirectoryOrJunction =
      ((fileAttributes & FILE_ATTRIBUTE_DIRECTORY) != 0) ||
      ((fileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) != 0);

    if(!isDirectoryOrJunction) {
      throw std::runtime_error(
        "Could not create directory because a file with the same name exists"
      );
    }

  }
}

This code is free for the taking and you can use it however you want.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Visual Studio 2012 Express – Metro Only?

This post’s title says it all. I’ve just installed the Windows 8 Release Preview with Visual Studio 2012 RC. Just like in the previous release, Visual Studio 11 Beta, the Express edition does not contain any plain Win32 project templates, only ones for Microsoft’s new Metro UI.

This is a pretty scary situation for me: recently, the C++11 Standard was completed which finally makes threading in C++ bearable (when writing libraries, you no longer have to force a decision between Boost, TBB or POCO on your users). But C++11 threading was only added to Microsoft’s C++ compiler & standard library in Visual Studio 2012.

So unless Microsoft reverts the decision to no longer offer a free C++ compiler for Windows desktop applications, these are my options:

  • Buy Visual Studio 2012 Professional and exclude anybody out there who doesn’t want to shell out $500 from using my libraries. Make it harder to build any kind of team because there’s now a significant hurdle to entry. Probably be forced to modify 3rd party libraries in the future because Open Source projects will no longer test compilation of their code with Visual Studio.
  • Keep Visual C++ 2010 Express and either write my own threading wrapper (increasing dependencies of my libraries from 0 to 1) or just go with Boost and forget the rest.
  • Switch to Eclipse CDT + MingW and build my Windows desktop applications this way. That would give me C++11 threading (at least I suppose so – I couldn’t get it to work, see screenshot) but I’d have to always port anything I write to GCC (of which MingW is a port). Could be a good thing, since it lowers the barrier to port my stuff to Linux and Android.

Update: if you want to weigh in, I’ve been so bold and filed a bug report on Microsoft Connect: Support C++ Desktop Applications in Visual Studio 2012 Express.

Update2: the disaster is averted! Today, the Visual Studio blog announced that Microsoft has given in and will be offering two Express editions: The already known Visual Studio 2012 Express and, Visual Studio 2012 Express for Windows Desktop. That means it’s safe to use C++11 across the board now. GCC 4.6 has it, the modded Android NDK has it, and Visual C++ 2012 has it!

Read More