# DDI Compendium API gotchas



## Asmor (Aug 7, 2009)

I'm doing some work with the Compendium API, and finding that the documentation is not just... poor, but it's downright inaccurate in places. Working on doing filter select for monsters, and here's some things I've learned which I hope will help you.

1. The documentation says that non-specified filters must be "NULL". This is incorrect. They should be "null"-- lower case. This is one of the few places I've found where case matters.

2. When searching for monsters, the minimum and maximum xp cannot be "null". I've been using values of 0 and 1000000 respectively.

3. If you're using any sort of automation to pick up the interface, the API itself has the incorrect address listed. It lists "http://www.wizards.com/CompendiumSearch.asmx/[METHOD]" but it should be "http://www.wizards.com/dndinsider/compendium/CompendiumSearch.asmx/[METHOD]".

4. It says that several parameters are optional. For example, with a KeywordSearch, Tab is listed as being optional. This is technically true, but you still need to pass a Tab parameter-- just leave it empty.

5. Since releasing the API documentation, they've added a new required parameter called "NameOnly." It doesn't appear to do anything (I've tried setting it to both True and False, and noticed no difference). I recommend setting it to false. This parameter is required for KeywordSearch and KeywordSearchWithFilters

If anyone else has caught anything or just has any tips, please post 'em! This is my first time working with someone else's API and with SOAP, so I'd love to pick people's brains.

Oh, one thing that might not be obvious... If you're doing something in JavaScript, many recent browsers such as Firefox 3.5 have built in protections against cross-domain scripting which prevent a page on, say, asmor.com from loading data from, let's say, wizards.com. One way to get around that is to use a proxy on your webserver... I did some searching and found a PHP proxy specifically for these purposes which grabs the external data and returns it.


----------



## Asmor (Aug 7, 2009)

Woops, just found another one. Minimum and Maximum levels also can't be null. I'm using 0 and 100 as defaults when not specified.


----------



## malraux (Aug 7, 2009)

For numeric fields, typically -1 is the effective null command.  The nameonly flag should be there to only search in the names and not in the rest of the text, I haven't checked it to see if it works, and just set mine to always false as well.

You should really get a packet sniffer to check the stuff being sent from the official compendium app.  It's more accurate than using the official documentation.


----------



## malraux (Aug 7, 2009)

Ok this one has been bugging me.  How the hell does the compendium store data about trap types?  It obviously knows traps from hazards, but the xml doesn't store that data in any apparent way.  Looking through the java of the compendium, it doesn't have a list of that info anywhere, which would defeat the point of storing stuff in xml files anyway.  This is seriously bugging me.


----------



## malraux (Aug 7, 2009)

Oh, and not to overpost or anything, but here is the current version of my compendium interface for the mac.  It should run on 10.4 and up.  My development of it at the moment is stalled as I'll need to rework the interface because apple doesn't let you do dumb stuff with tabs (ie create multiple rows of tabs).

The main bug issue right now that's obvious is that the clear button doesn't do anything.  Lesser issues are that I need to clean up the column sorting.  The app also seems to take a while to load, which is cause by having to get data from the wizards website before opening the window.  But for mac use, I've found it to be much faster than using the web based wizards one.

Feedback is appreciated.


----------



## mpattee (Aug 12, 2009)

malraux said:


> Oh, and not to overpost or anything, but here is the current version of my compendium interface for the mac.  It should run on 10.4 and up.  My development of it at the moment is stalled as I'll need to rework the interface because apple doesn't let you do dumb stuff with tabs (ie create multiple rows of tabs).
> 
> The main bug issue right now that's obvious is that the clear button doesn't do anything.  Lesser issues are that I need to clean up the column sorting.  The app also seems to take a while to load, which is cause by having to get data from the wizards website before opening the window.  But for mac use, I've found it to be much faster than using the web based wizards one.
> 
> Feedback is appreciated.




I was curious if you're developing your interface into the soap calls as a separate library? If so have you thought about making that portion of the app available? I'm currently working on a few iphone apps that this would be quite helpful with. Also if you're interested I could probably help a bit on your desktop app as well. My mind is racing a bit right now since I just came across the API options for the DDI compendium.


----------



## mpattee (Aug 12, 2009)

Asmor said:


> 5. Since releasing the API documentation, they've added a new required parameter called "NameOnly." It doesn't appear to do anything (I've tried setting it to both True and False, and noticed no difference). I recommend setting it to false. This parameter is required for KeywordSearch and KeywordSearchWithFilters




Setting NameOnly to true makes it so it only searches in the name field for the keywords.


----------



## Asmor (Aug 12, 2009)

mpattee said:


> Setting NameOnly to true makes it so it only searches in the name field for the keywords.




Ah, cool. Thanks!


----------



## malraux (Aug 13, 2009)

mpattee said:


> I was curious if you're developing your interface into the soap calls as a separate library? If so have you thought about making that portion of the app available? I'm currently working on a few iphone apps that this would be quite helpful with. Also if you're interested I could probably help a bit on your desktop app as well. My mind is racing a bit right now since I just came across the API options for the DDI compendium.




All I'm doing is an http request and then getting the result as an XML document.  Here's the relevant section of code.

```
NSMutableString *string1 = [NSMutableString stringWithFormat:@"http://www.wizards.com/dndinsider/compendium/CompendiumSearch.asmx/KeywordSearchWithFilters?Keywords="];
		[string1 appendString:[[searchText stringValue] copy]];
		[string1 appendString:[self filterSearchString]];
		searchResultsFromServer = [[NSXMLDocument alloc] initWithContentsOfURL:[NSURL URLWithString:[string1 stringByAddingPercentEscapesUsingEncoding:NSUTF8StringEncoding]] options:NSXMLDocumentTidyXML error:&err];
```
Unfortunately, everything I've done is effectively useless for iphone development.  The iphone doesn't use the NSXMLDocument class, which means all my code would need to be rewritten to immediately parse the incoming XML.  Beyond that, the code is as simple as could be.


----------



## mpattee (Aug 13, 2009)

What about the calls to the detail of the items. It appears that those are coming back as pre-formatted html? I wish they'd open that up with authentication and return the information in xml.

My excitement dwindled when I how they were handling that part. 

I'm mostly done with another app that is using soap on the iPhone. Parsing out the xml wasn't too bad with the right libraries to help out. If you'd still ike some help with your app let me know.


----------



## Asmor (Aug 13, 2009)

There's a third article on the API promised which is supposed to handle authentication and, they were a bit vague, but it sounded like it might have info on getting the "raw" data on the various rules components.

Of course, no telling when that's going to happen.


----------



## malraux (Aug 13, 2009)

mpattee said:


> What about the calls to the detail of the items. It appears that those are coming back as pre-formatted html? I wish they'd open that up with authentication and return the information in xml.



Yeah, all I do is open a window with a Webview inside with the address of the item.  It isn't fancy, but its all the online compendium app does as well.


----------



## catsclaw227 (Aug 13, 2009)

Asmor said:


> There's a third article on the API promised which is supposed to handle authentication and, they were a bit vague, but it sounded like it might have info on getting the "raw" data on the various rules components.
> 
> Of course, no telling when that's going to happen.



I;'ve been waiting in this as well.

I currently have a WPF applet that makes the calls and returns the HTML for teh individual item.

What I want, is for the individual item to be returned as XML based around an XSD. This would make return of the data and deserializing it into an object a trivial task. The object would be the pile of gold, assuming that it specifically defined elements like a monsters attacks, bonuses, defence attacked, etc..


----------



## Mercule (Aug 24, 2009)

I can't get anything out of this, even hitting it through IE.

Granted, I'm an intranet developer, so remote security is a bit out of my normal range, but this is very vexing.  I can't even get a reference to this through Visual Studio.  Anyone want to through me a bone -- just a single good connection would be awesome?


----------



## Asmor (Aug 24, 2009)

Err... Could you be a bit more descriptive with your issue? E.g. what, exactly, do you mean by anything? Authentication? Running searches? Connecting to the API?

Since you mention Visual Studio, I'm going to go on a wild leap of intuition here and guess that you're trying to add the API as a service reference and it's not working. If that's the case, as I mentioned in my original post the service itself is misreporting its location (see point 3 in my first post) and so you've gotta change that...

I'd give you more details, but I can't remember how I got it working in the first place, and can't recreate it at the moment.  But there was some file hidden from view in the IDE, but visible in the normal file structure, an xml file with a weird extension. Near the bottom of that file were the URLs for the calls, and that's where you'd change them.


----------



## Mercule (Aug 24, 2009)

Sorry, I was kinda tired and frustrated last night.

I VS, I'm trying to add a web reference using the address you posted.  I get a 403 exception, so I assume I somehow need to authenticate against my DDI subscription when having VS create the proxy class.  As I said, I'm used to developing internal web services and authenticating against Windows tokens, so I'm figuring it's a pretty dumb question, overall.

The other thing I'm trying, without success, is to hit the asmx through IE and run the methods directly.  For instance, I'd expect to be able to hit CompendiumSearch Web Service, type "Feat" in the box, click "Invoke", and get a chunk of XML back on the screen.  Instead, I get a yellow ASP.NET error page that doesn't tell me anything of value.

Basically, I know there's something stupid I'm missing about invoking the web service and hoping someone whose done it successfully can tell me what.  I'm a fairly experienced .NET developer, but web services are something I usually use only when I need to.

Edit: hmm... ENWorld auto-converted my URI to a title.  The link is still valid, though.


----------



## Asmor (Aug 25, 2009)

Yeah, it doesn't work directly from the webpage for exactly the reason I stated... It's misconfigured. So when you submit the request, it's going to the wrong address.

Unfortunately, authentication is an issue I know pretty much nothing about. That said, I haven't needed to do any authentication to run queries in the past. I haven't touched the API in a while, though, so maybe they've changed it.


----------



## Mercule (Aug 26, 2009)

You mentioned documentation.  Is there anything more than the two articles?


----------



## Asmor (Aug 26, 2009)

Negative


----------



## mnology (Nov 13, 2009)

*How to log in to the API*

First, your programming language of choice should have some API for automating interacting with webpages intelligently.

I'm a Perl programmer so I'm using WWW::Mechanize. It manages a www session and cookies for me and allows me to select a form, input fields and press form buttons on webpages really easily. I know ruby has a mechanize, and there's probably a python port too. I avoid microsoft languages & java like the plague. That said, this information should be able to get you on the right track no matter your language.

We're going to start with the base D&D Insider page, because this is where the login form is: Dungeons & Dragons Roleplaying Game Official Home Page

On this page we want to work with the 'aspnetForm'
Username field: 'ctl00$ctl00$ctl00$ctl00$SiteContent$DnDContent$ctl00$UserName'
Password field: 'ctl00$ctl00$ctl00$ctl00$SiteContent$DnDContent$ctl00$UserPassword'
Login button: 'ctl00$ctl00$ctl00$ctl00$SiteContent$DnDContent$ctl00$LoginButton'

With WWW::Mechanize I can just populate the appropriate fields and say $mech->click($login_button) and I'm logged in.

The cookie for testing whether or not you are logged in is '.ASPXAUTH'

You can logout by going back to the original page, and if you're logged in the Login button ('ctl00$ctl00$ctl00$ctl00$SiteContent$DnDContent$ctl00$LoginButton') is the same one to 'click' to logout. When you do this the '.ASPXAUTH' cookie is cleared.

When logged in using the same mechanize instance, I can $mech->get($compendium_item_uri) and I can get the content of a compendium uri and get the actual results, and not the pesky log in content.

Check out my WIP: mnology's perl-ddi-compendium at master - GitHub for some real code.


----------

