Daggerfall Workshop

Exploring Daggerfall

Daggerfall Workshop RSS Feed
 
 
 
 

A Few Quick Things

Just a quick post to outline the next round of updates coming when I return to the Workshop.

  • The Daggerfall Connect project format has been changed back to Visual Studio 2008 / C# Express 2008. This decision was made to support a wider range of people. Naturally, you can still upgrade the project to a 2010 version after you download.
  • Daggerfall Connect project has been created on Google Code. SVN access will be ready as soon as I can shift over at my end. All online documentation and tutorials will be moved to the project wiki.
  • I am retiring the “Workshop Demo Engine” (based on Ogre) as the standard 3D engine for Daggerfall Connect. All future Daggerfall Connect tools and demos using 3D graphics will be based on XNA.
  • The next release of Daggerfall Connect will introduce an XNA tutorial series covering how to render individual models and blocks, through to entire cities and dungeons.
  • Map support will be finalised in the next release of Daggerfall Connect.

I am also pleased to announce that I have re-started Daggerfall Scout as my next project. I have a new vision of this tool as an Atlas-like explorer covering the major points of interest in Daggerfall. I will post more details on this project as soon as I can.

Daggerfall Connect Source Code

The current stable sources for Daggerfall Connect can be downloaded below. This archive also contains sources for Daggerfall Imaging 2 and several tutorials.

Download Daggerfall Connect Sources 0.4.7 (Self-extracting archive)

All you need to build is:

SVN access will be available soon, in addition to other open-source project features. I will post more on this when I am able.

Australia is heading into a new financial year. As is usual at this time of year, I need to step away from the Workshop for a while. I look forward to posting more updates when I return.

Daggerfall Imaging 2 Released!

I’m pleased to announce that Daggerfall Imaging 2 is now available for download. This version is a major upgrade to Daggerfall Imaging, with the following key changes:

  • An improved, fully visual user interface with thumbnail previews.
  • A flow layout image browser with image selection, mouse-over properties, and clipboard support.
  • Bulk exporting with a tightly-integrated export manager.
  • Custom descriptions for image files.
  • Written entirely in C# using the Daggerfall Connect Library.

Download Daggerfall Imaging 2

Download Manual (In PDF format. Requires Adobe Reader)

I have decided to release all the source code for my new tools (currently Daggerfall Connect and Daggerfall Imaging 2) under the free MIT License. More details will be released in the coming weeks.

   
 

There are a few limitations with the first release of Daggerfall Imaging 2 that will be corrected in the next version (2.1). These are:

  • Limited GIF support. Poor palette matching, no transparency or animation.
  • Image Browser renders flow layout only. A print layout is planned for 2.1.
  • No printing or document export support at this time.

I look forward to hearing your feedback about Daggerfall Imaging 2. If you encounter any bugs, or have any suggestions, please contact me.

Daggerfall Imaging 2 Export Manager

I’ve received a couple of emails from people asking me if Daggerfall Imaging 2 can export to more than just GIF files. This isn’t the first time I’ve heard this question. Many seem to have missed the fact you could do this in Daggerfall Imaging 1.0.

I think the reason for this is the old web-like interface. Users discover the right-click copy method and don’t notice the “Export” icon on the toolbar that allows you to export to a wide variety of formats.

Considering this, I’ve decided to overhaul how you export files in Daggerfall Imaging 2. Rather than use a modal popup dialog, this is now tightly integrated into the main view via a sidebar that can be toggled open or closed. The screenshot below shows DF Imaging 2 in near-complete form.

 

The left-hand pane shows thumbnails of image files by library (textures, img, cif, etc.). You can mark image files for export by putting a tick in the check box of any item. A right-click context menu allows you to check all, check none, or invert checked items. You can also set your own descriptions for each image file, which is stored in an XML database.

The middle pane is the Image Explorer; a flow view for viewing images up close. You can toggle animation, transparency, and zoom in or out. The status bar tells you about the image you are hovering over. Another right-click context menu allows you to copy images to the clipboard.

The right-hand pane is the Export Manager. This pane allows you to set an export format and destination, along with a few options. The Export Preview tree previews the folder structure created by the exporter. This structure will change based on your destination folder, chosen image library, and selected format. The goal is to keep your exports neatly organised. These settings are remembered each time you open DF Imaging 2. This tight integration should make it easy to export as many (or as few) files as you need.

Everything is coming together nicely. I just need to complete the new export manager and DF Imaging 2 will be ready for release. Hopefully this is something I can finish over the weekend.

Daggerfall Imaging 2 Not Far Away

I’m almost done with the first release of DF Imaging 2. I’m just bug-catching and putting a little polish on some things. I should have it ready for download in the next week.

Edit: Just a quick update. I’ve been unable to work on Daggerfall Imaging 2 for the last few days due to real-life issues. I’m back to working on it now.

DFConnect Rename & Update

I’m gearing up for a new release of DFConnect, which has been renamed to Daggerfall Connect. I feel this is more in line with the naming convention of my other tools and avoids any potential misunderstanding with a trademarked property also called DFConnect.

The next build is a major update across the entire namespace. I have temporarily taken down all Daggerfall Connect pages while I update the documents, tutorials, FAQ, etc. These should be back in a couple of weeks.

To summarise, the following changes have been made to Daggerfall Connect:

  • DaggerfallConnect root namespace now contains classes and structures emitted back to applications, such as DFBlock and DFMesh. These were previously in the Arena2 namespace. The reasoning is to keep ”low-level” classes in Arena2 and “high-level” classes in the root namespace.  The Arena2 namespace now only contains classes for opening files in the Arena2 folder.
  • The Utility namespace has been added. This is for mostly internal classes used by Daggerfall Connect, but may contain public classes at a later date.
  • The first reader class has been added. ImageFileReader is a vastly simplified way of opening any Daggerfall image file.
  • All image classes are now based on abstract base class DFImageFile. This allows ImageFileReader to handle all image files in a standard way and return a standard object back to applications regardless of the image file type originally opened.
  • Casing changed from lower camel casing (e.g. getFrameCount) to upper camel casing (e.g. GetFrameCount). This is to be more in line with .NET Framework conventions.

I will post more on this after Daggerfall Imaging 2 is released.

Daggerfall Imaging 2 – User Interface

This post is about a few of the UI enhancements going into Daggerfall Imaging 2. Not exactly gripping stuff, but I know many people are still using the old DF Imaging and may be interested in where I’m going with the new version.

When you open the old version of Daggerfall Imaging, you’re greeted with the following dense menu of image files to choose from:

Talk about a wall of text! Most people only fire up Imaging once they’ve found the file they want in Daggerfall Explorer first, and even then it’s easy for the eye to get lost. Things don’t get any better once you’ve opened an image file, either. You’re limited to simple “next-file” and “previous-file” buttons or doomed to return back to the mess above and start again. Let’s face it, Imaging is a nice exporter with a rubbish UI. And haven’t people let me know it; I’ve received dozens of emails over the years condemning the user interface.

So while finishing up image handling in DFConnect, I thought: “what the hell, let’s rewrite the old mongrel.” It was a perfect opportunity to improve on an old tool and catch some bugs in my new library.

Searching through images should be fun and easy, and the best way to accomplish that is to let the images do all the talking. It’s much easier to find what you want from thumbnails than plain-text filenames, so the first change was to put in a nice thumbnail viewer.

 Thumbnails Detail Tiles

The thumbnails are housed in a sizing pane and can be rendered in a few different ways. Currently, there are two formats for browsing Daggerfall’s image files. The Thumbnails layout is just a preview of each image file. You’ll notice these thumbnails are mini views of the entire file, not just a single image. Think of them as thumbnails inside thumbnails. The Detail Tiles layout has the same thumbnails, but with additional information about the image file itself.

This simple change makes browsing into image files a lot more visual and fun. I never feel “lost” among the files, something I felt all the time in the old Daggerfall Imaging.

For those of you following DFConnect, the thumbnails are generated by a new method called GetPreview() that renders an image file onto a managed Bitmap. You can specify the dimensions of the resultant Bitmap emitted by this method, allowing you to fit more/less images into a single preview. All DF image classes support this method.

New Exploring Tools

I’ve mentioned in the past that I plan on releasing standalone exploring tools based on DFConnect. The first of these tools is under development now, and proceeding rapidly. Below are a couple of screenshots of Daggerfall Imaging 2.0.

   

Daggerfall Imaging has always been the least favourite of my old tools. Despite solid exporting features, it was let down by many aspects of the user interface. To date it is the only tool I have received negative feedback about. For these reasons it became the first program I wanted to revisit as DFConnect matures.

I plan on releasing the first version of Daggerfall Imaging 2.0, complete with source code, before the end of May. This will coincide with an update to DFConnect. The goal is to create something useful and fun while honing DFConnect through actual use.

Time permitting, I hope to turn out new small-to-medium exploring tools every few months. This will give everyone something to play with and demonstrate what DFConnect is capable of.

Man, It’s Dusty In Here

Time to pick up a broom and get back to work. I hope everyone has been keeping well while I’ve been away.

I will be getting back into regular updates soon. After breaking from Daggerfall for a while, I’m keen to get back into the swing again. I have an updated version of DFConnect to post and more news on those mini tools I spoke about a few posts back.

Thanks to everyone that has written to me over the last few months. I’m still very much alive and haven’t abandoned the Workshop by a long shot. More to come soon.

DFConnect Library 0.4.5

DFConnect Library 0.4.5 is now available for Download. Please check the Issues page for current known issues, and the System Requirements page for help getting started.

The key feature of this build is RMB and RDB blocks are almost fully supported by the library. This release includes an RMB Block Viewer Demo (binary and source code included with library download). The control scheme of this tool follows the RDB viewer:

RMB (Outdoor) Block Viewer

  • Click and hold the left mouse button while moving the mouse to look around in a first-person manner.
  • Click and hold the right mouse button while moving the mouse forward and backwards to move along the camera’s facing vector.
  • Click and hold the middle mouse button while moving the mouse to go up, down, left, and right relative to the camera’s facing vector.
rmb viewer 1 rmb viewer 2
rmb viewer 5 rmb viewer 3
rmb viewer 7 rmb viewer 4

The upcoming 0.5.0 build will introduce Maps.bsa support, along with climate processing (i.e. using correct textures for swamp, desert, etc.). This build will also include a simple map viewer demo similar to early screenshots of Daggerfall Scout.

The 0.5.5 build will be a bug-fixing and polishing build. If you have any problems with DFConnect not mentioned on the Issues or Troubleshooting pages, please Contact me so I can address them.

Starting with 0.6.0, I will be adding more usability features to the root DFConnect namespace. While the individual classes in the DFConnect.Arena2 namespace greatly simplify access to Daggerfall’s files, the addition of custom readers will make this easier.

I have started work on standalone applications that demonstrate advanced use of DFConnect. The plan is to release these separately to the DFConnect Library as new exploring tools. Details of these tools will be released shortly after the 0.6.0 milestone.

Improving UV Generation

I recently engaged in a fruitful conversation on the DaggerXL forums with another DF hacker called DigitalMonk, whose specialty is Daggerfall’s textures. His interest was piqued when I made an informal comment regarding UV generation. After bouncing a couple of posts off each either, DigitalMonk’s thoughts and observations resulted in improvement to my current method of UV generation. He also made a discovery that will result in yet another improvement down the track. If you want to track the whole conversation, read back from here. I will summarise the outcome below so you don’t have to re-read the whole thread unless you want to.

Anyone who used Daggerfall Explorer would have noticed some 3D objects have skewed textures. At the core of this problem is how Daggerfall stores UV coordinates (UV mapping is a way of describing how textures are painted on a mesh). Rather than storing a value from 0.0 to 1.0 for every point, Daggerfall only describes UV coordinates for the first three points of any face (and optionally on the fourth point, but research has shown DF does not use this value). This probably doesn’t sound so bad, triangles only have three points right? Unfortunately Daggerfall doesn’t use triangles to describe 3D objects, it uses ngons that can exceed a dozen points per face. With only the first three UV coordinates to work from, the problem is how do we calculate UVs for the remaining points? Check out the main article on the UESP for more information on the problem and the solution we’ve been using up until now. You get some feeling from this short article on how much special handling is needed to cover all possible faces.

The problem with this solution is the constructed matrix can generate UVs that look a little weird, even when the determinant for that linear equation believes the face should be solvable. There are also multiple linear equations based on whether the face is coplanar to XZ, XY, YZ, etc. When UV generation failed, branching logic was needed to post-fix the UVs for certain faces, or skip the matrix generator for another technique, or just slap the correct UVs on directly. All of this made UV generation a bloated exercise that was tricky to debug. Fixing one problem would often break something else. All in all, there are three classes of texture problems:

  • Type 1. These are faces for which UVs cannot be generated using the existing linear equations, or are generated incorrectly.
  • Type 2. These are faces with bad source UVs (i.e. the coordinates are incorrect in Daggerfall’s files). Check out the eastern side of Wayrest Castle in-game for an example of this problem. This class of problem is visible inside the game.
  • Type 3. I used to think these were Type 2 problems, but DigitalMonk discovered these source UVs actually have extra bits packed into the coordinate. By removing these bits, the UV coordinate works as expected. This class of problem is not visible in the game, meaning Daggerfall uses these bits for some purpose, or at the very least just strips them out.

After my conversation with DigitalMonk, I’ve made some improvements to my UV generation that removes all Type1 problems with no special handling required. I managed this after realising UV generation is really just a 2D problem in 3D space. All of those extra linear equations were only needed due to the extra degree of freedom. By moving all the points into 2D first, I would only need the single XY linear equation to solve for all possible faces.

This worked perfectly, and allowed me to remove all my special handling. Below are a couple of screenshots from the test block viewer. The screenshots on the left demonstrate some problem faces with all special handling disabled. You don’t see these problems in the RDB Block Viewer demo included with DFConnect because of that special handling. On the right is the same scene using the new UV generator, with absolutely no special handling. All UV coordinates “just work”.

uv fixing 1 uv fixing 2
uv fixing 3 uv fixing 4

The next step will be to strip those extra bits for Type3 problems, and eventually write a simple UV patching function for the worst Type2 problems. Once this is completed, UV generation in DFConnect will be almost perfect.

I can’t thank DigitalMonk enough for taking the time to clarify the various UV problems and help me reach a better solution. Cheers mate.

DFConnect Library 0.4.0

The 0.4.0 library release is now ready for download. The Tutorials pages have been updated due to a few namespace changes. I have also added a System Requirements page.

The demos are very basic examples of how to render content out of Daggerfall’s files. Demos will be added for all major content types as the features are added to DFConnect.

I need to add pages specific to each demo. Until then, the controls for the two initial demos are:

Mesh Viewer

  • Click and hold the left mouse button while moving the mouse left and right to rotate the mesh.
  • Click and hold the right mouse button while moving the mouse forward and backwards to zoom in and out.

RDB (Dungeon) Block Viewer

  • Click and hold the left mouse button while moving the mouse to look around in a first-person manner.
  • Click and hold the right mouse button while moving the mouse forward and backwards to move along the camera’s facing vector.
  • Click and hold the middle mouse button while moving the mouse to go up, down, left, and right relative to the camera’s facing vector.

The FAQ and Issues pages have been posted. A Troubleshooting page has been added for common problems relating to things like build settings and dependencies.

The 0.4.5 release should be ready this weekend. This build will support RMB (Outdoor) blocks. From that point, I will start adding map and world reading for the 0.5.0 build.