Bea Stollnitz

Blog

Books

Books I tech reviewed and recommend

Sorry I can't reply to every email and comment. If you need a fast response, try the WPF Forum or Silverlight Forum.

All original code referenced in this blog is licensed under the Open Software License version 3.0.

April 4, 2010

How can I update LabeledPieChart to use the latest toolkit?

A few months ago I blogged about a behavior that adds labels to a pie chart in Silverlight or WPF. I wrote a post showing the usage of that behavior, another one explaining the implementation details of the WPF version, and another one explaining issues encountered when porting it to Silverlight. Since those posts were written, a new version of the charting APIs came out with improvements that help simplify the labeled chart code. By popular request, this blog post explains how I updated the WPF and Silverlight pie chart label behavior to take advantage of the latest Silverlight Toolkit and WPF Toolkit.

The Chart class is no longer sealed

When the pie chart label behavior was originally written, the Chart class was sealed. As a workaround, I created a LabeledPieChart custom control that derived from Control and contained a Chart with a PieSeries in its template. This inner Chart also included in its template an extra panel where the labels were added. The big drawback of this approach was the fact that I had to re-expose all the interesting properties of Chart and PieSeries in the custom control, and make sure that the new properties were updated when the original ones changed.

Now that Chart is no longer sealed, I am able to implement LabeledPieChart by deriving from Chart directly. This class is greatly simplified, since I no longer need to re-expose any properties. The only reason it exists is so I can give it a template containing the panel where the labels will be added.

    [TemplatePart(Name = "LabelArea_PART", Type = typeof(PieChartLabelArea))]
    public class LabeledPieChart : Chart
    {
        static LabeledPieChart()
        {
            DefaultStyleKeyProperty.OverrideMetadata(typeof(LabeledPieChart), new FrameworkPropertyMetadata(typeof(LabeledPieChart)));
        }
    }

The PieSeries class is no longer sealed

In my earlier solution, the code that added the labels to the panel was implemented using a behavior – PieLabelBehavior – which was attached to each PieDataPoint. The XAML that attaches the behavior to its target was found in the LabeledPieDataPointStyle, in generic.xaml.

Now that PieSeries is no longer sealed, there is no need to use a behavior any more. Instead, the code that adds the labels is now located in a new class (LabeledPieSeries) that derives from PieSeries. The method below shows the logic that adds labels corresponding to each data point.

    protected override void OnAfterUpdateDataPoints()
    {
        LabeledPieChart chart = this.SeriesHost as LabeledPieChart;
        if (chart != null)
        {
            Canvas labelArea = chart.Template.FindName("LabelArea_PART", chart) as Canvas;
            if (labelArea != null && this.ItemsSource != null)
            {
                foreach (object dataItem in this.ItemsSource)
                {
                    PieDataPoint pieDataPoint = this.GetDataPoint(dataItem) as PieDataPoint;
                    if (pieDataPoint != null)
                    {
                        this.AddLabelPieDataPoint(pieDataPoint, labelArea);
                    }
                }
            }
        }
    }

(The code for AddLabelPieDataPoint hasn’t changed from my earlier code, so I don’t show it here.)

In addition, PieSeries contains three properties that were previously in the LabeledPieChart custom control: PieChartLabelStyle, PieChartLabelItemTemplate and LabelDisplayMode. These are the only new properties needed by the code that adds labels. Since the ItemsSource property is in PieSeries, it makes sense to put these other properties in PieSeries, too.

The fact that I no longer need to attach a dependency property to each data point allowed me to remove the corresponding XAML in generic.xaml.

Usage

Now that we have a LabeledPieChart class that derives from Chart and a LabeledPieSeries class that derives from PieSeries, the usage of the control has changed a bit. Here’s some of the XAML from the project that I link to at the end of this post:

    <customControls:LabeledPieChart
        x:Name="labeledPieChart"
        Title="Population of Puget Sound Cities"
        Height="500" Width="700"
        Grid.Row="3"
        BorderBrush="Gray"
        >
        <customControls:LabeledPieChart.Series>
            <customControls:LabeledPieSeries
                x:Name="labeledPieSeries"
                ItemsSource="{Binding}"
                IndependentValuePath="Name"
                DependentValuePath="Population"
                IsSelectionEnabled="True"
                PieChartLabelStyle="{StaticResource pieChartLabelStyle}"
                PieChartLabelItemTemplate="{StaticResource pieChartLabelDataTemplate}"
                LabelDisplayMode="Auto"
                />
        </customControls:LabeledPieChart.Series>
    </customControls:LabeledPieChart>

As you can see, instead of setting a few attached properties (to add the behavior), now you use the LabeledPieChart and LabeledPieSeries classes directly. There is one big advantage to this technique: now you can use all properties available in Chart and PieSeries to customize your chart (not just the ones that were re-exposed in the LabeledPieChart in my previous implementation).

The fact that Chart and PieSeries are no longer sealed allows me to greatly simplify the source code (and XAML) of the labeled pie chart. In addition, it provides more flexibility for the developer to customize the chart and series.

Download the WPF project (built with .NET 3.5).

Download the Silverlight project (built with Silverlight 3).

Posted by Bea under Silverlight, WPF | Comments (2)

March 20, 2010

How can I sort a hierarchy?

Today’s blog post discusses how you can sort data items at each level of a hierarchical user interface in WPF and Silverlight. Instead of just giving you the final solution, I will first show a few approaches that you might expect to work, and I’ll explain why they don’t work. All of the approaches will use a single data source: a list of State objects, each of which contains a list of County objects, each of which in turn contains a list of City objects. I’ll assume you’re familiar with the sorting abilities of CollectionViewSource (see this post) and binding to hierarchical data (see this post).

WPF

Attempt 1

The first approach most people try is to add a CollectionViewSource (with a SortDescription) for each level of the hierarchy directly to the root element’s resources, and then bind the ItemsSource property of each HierarchicalDataTemplate to the corresponding CollectionViewSource. The XAML below shows the resources, assuming the root element’s DataContext has a property called “States” that returns a collection of State objects.

This approach doesn’t work — the names of the states show up, but without any counties or cities. That’s because all three CollectionViewSources have the same DataContext: the data source with the States collection. The DataContext is not determined based on where a CollectionViewSource is consumed, as you might expect — it depends only on where the CollectionViewSource is declared. Visual Studio’s output window shows two binding errors indicating that there are no Cities or Counties properties:

System.Windows.Data Error: 39 : BindingExpression path error: ‘Cities’ property not found on ‘object’ ”DataSource’…

System.Windows.Data Error: 39 : BindingExpression path error: ‘Counties’ property not found on ‘object’ ”DataSource’…

Attempt 2

With this in mind, it’s natural to look for a solution where the CollectionViewSources are added directly to the HierarchicalDataTemplates, so that each one has a DataContext that is set to the appropriate data item. This approach adds a CollectionViewSource to the Resources section of each HierarchicalDataTemplate:

This also doesn’t work. Again, the states appear but the counties and cities don’t. Although each instance of the HierarchicalDataTemplate has its DataContext set to the appropriate data item, the resources section does not share that same DataContext. This happens because the template’s resources are shared among all template instances (and each template instance has a different DataContext). The output window in Visual Studio shows two error messages indicating that the Cities and Counties bindings are not inheriting any DataContext:

System.Windows.Data Error: 2 : Cannot find governing FrameworkElement or FrameworkContentElement for target element. BindingExpression:Path=Cities; DataItem=null…

System.Windows.Data Error: 2 : Cannot find governing FrameworkElement or FrameworkContentElement for target element. BindingExpression:Path=Counties; DataItem=null…

Attempt 3

Resources within the HierarchicalDataTemplates don’t have the correct DataContext, so we need to find a way to add each CollectionViewSource to a property of the appropriate HierarchicalDataTemplate. We can’t set the HierarchicalDataTemplate’s ItemsSource property directly to the CollectionViewSource because the CollectionViewSource does not implement IEnumerable , but we can use a binding:

This approach also fails to show counties and cities because the Binding in the CollectionViewSource’s Source property does not have a DataContext. The Binding used as the HierarchicalDataTemplate’s ItemsSource doesn’t propagate its DataContext to the CollectionViewSource within it. The output window in Visual Studio shows the same error messages as in Attempt 2.

Attempt 4

Since CollectionViewSource doesn’t seem to be working, the next logical step is to do the sorting ourselves in code, and to place this code in a Converter:

    <HierarchicalDataTemplate x:Key="StateTemplate" DataType="{x:Type local:State}" ItemsSource="{Binding Counties, Converter={StaticResource SortCountiesConverter}}" ItemTemplate="{StaticResource CountyTemplate}">
        <TextBlock Text="{Binding StateName}" />
    </HierarchicalDataTemplate>

    public class SortCountiesConverter1 : IValueConverter
    {
        public object Convert(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
        {
            IEnumerable<County> counties = value as IEnumerable<County>;
            if (counties != null)
            {
                return counties.OrderBy(county => county.CountyName);
            }
            return null;
        }

        public object ConvertBack(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
        {
            throw new NotImplementedException();
        }
    }

This approach works if your data source doesn’t change. It sorts each level of the hierarchy when the application starts, but if you add items to the hierarchy later, those changes are not reflected in the UI. This is because the code uses Linq to do the sorting and returns a new (sorted) collection. The code doesn’t include any mechanism to re-sort the collection and propagate changes to the UI when new items are added to the original collection.

Solution

This brings us to the final approach, and to a working solution. Instead of using Linq to return a new collection, we can simply return a sorted WPF view that points to the original collection:

    public class SortCountiesConverter2 : IValueConverter
    {
        public object Convert(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
        {
            IEnumerable<County> counties = value as IEnumerable<County>;
            ListCollectionView lcv = (ListCollectionView)CollectionViewSource.GetDefaultView(counties);
            lcv.SortDescriptions.Add(new SortDescription("CountyName", ListSortDirection.Ascending));
            return lcv;
        }

        public object ConvertBack(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
        {
            throw new NotImplementedException();
        }
    }

Since we return a CollectionView that wraps the original collection, any changes in the collection are reflected in the UI.

Although we couldn’t solve the problem entirely in XAML, the code we ended up with is very simple and achieves the desired result while taking source changes into account.

Silverlight

In Silverlight, elements added to the resources of a parent element don’t inherit the DataContext of that parent element. For this reason, it doesn’t make sense to consider attempt 1 described above as a possible approach for Silverlight –none of the CollectionViewSources would have a DataContext and you would get an AG_E_PARSER_BAD_PROPERTY_VALUE runtime error.

Attempt 2 also doesn’t apply to Silverlight because templates do not have a Resources section in Silverlight. Again, you would get an AG_E_PARSER_BAD_PROPERTY_VALUE runtime error.

Attempt 3 makes sense to consider in Silverlight, but it won’t work for the same reasons as WPF. Except in Silverlight, instead of getting a binding error in the output window, you would see a AG_E_PARSER_BAD_PROPERTY_VALUE runtime error, once again.

Attempt 4 also doesn’t work for the same reasons as WPF. It sorts the items correctly, but it doesn’t take into account any changes to the data source. I’ve included this approach in the Silverlight project associated with this post.

The solution that I showed for WPF also works for Silverlight with some minor changes. Silverlight’s CollectionViewSource doesn’t have a GetDefaultView static method, so we need to create a new CollectionViewSource and use its View property to get at the view in the converters. Here’s the Silverlight-compatible version of the code I showed above for WPF:

    public class SortCountiesConverter2 : IValueConverter
    {
        public object Convert(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
        {
            IEnumerable<County> counties = value as IEnumerable<County>;
            CollectionViewSource cvs = new CollectionViewSource();
            cvs.Source = counties;
            cvs.SortDescriptions.Add(new SortDescription("CountyName", ListSortDirection.Ascending));
            return cvs.View;
        }

        public object ConvertBack(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
        {
            throw new NotImplementedException();
        }
    }

Other minor changes had to be made to the Silverlight project. In particular, HierarchicalDataTemplate, TabControl and TreeView had to refer to the Silverlight SDK, since they’re not present in the main download. Also, Silverilght 3 doesn’t support implicit styles, so I used the toolkit’s ImplicitStyleManager to expand all TreeViewItems (you can read more about this technique here). Keep in mind that Silverlight 4 supports implicit styles natively, so pretty soon there will be no need to use ImplicitStyleManager any more.

If you have Silverlight 3 installed, you can see a running version of this example in its own page or embedded below:

Download the Silverlight project (built with Silverlight 3).

Download the WPFproject (built with .NET 3.5).

Posted by Bea under Silverlight, WPF | Comments (1)

January 22, 2010

How can I sort data virtualized items in WPF?

You saw in my last post how you can filter data-virtualized items by delegating the filtering operation to the server. In this post, I will show you how you can sort data-virtualized items on the server by interacting with the DataGrid UI. The code in this post extends the code in the filtering solution from my other post, so make sure you read that first.

As a reminder, the solution I showed in my earlier post exposes a stored procedure called “GetSortedFilteredCustomers” which allows us to sort and filter a subset of customers (we use indices to specify the subset). We can indicate the sorting we want by passing the SQL sorting syntax as a string to the data provider:

    string sortString = "CustomerSince DESC";
    customerProvider = new CustomerProvider(this.CustomerSinceDatePicker.DateFrom, this.CustomerSinceDatePicker.DateTo, sortString);
    AsyncVirtualizingCollection<Customer> customerList = new AsyncVirtualizingCollection<Customer>(customerProvider, pageSize, timePageInMemory);
    this.DataContext = customerList;

In this post I will show how you can construct a sort string that reflects the user’s interactions with the DataGrid UI. Here are the behavior requirements for this app:

Initially, the data is sorted by “CustomerSince”, in descending order. To inform the user of this fact, the “CustomerSince” DataGridColumn should display a triangle pointing down.

If the user clicks on another column, the data should be re-queried to sort based on the clicked column in ascending order. The DataGrid UI should reflect that by displaying an upward pointing triangle in the appropriate column.

If the user clicks on that column again, the data should now be sorted in descending order.

If the user presses “Shift” and clicks on several columns, the items should be ordered by all those columns, in the order they were clicked.

With these requirements in mind, I decided that I was going to keep a list of sort descriptions in my app. For each column the user clicks, I want to keep the property name associated with that column, the sort direction (ascending or descending), and a pointer to the actual DataGridColumn.

    private List<CustomSortDescription> sortDescriptions;

    public class CustomSortDescription
    {
        public string PropertyName { get; set; }
        public ListSortDirection Direction { get; set; }
        public DataGridColumn Column { get; set; }
    }

Now we need to be notified whenever the user clicks on a column for sorting. This can easily be done by listening to the “Sorting” DataGrid event.

In the handler for this event, we will add our custom sorting logic. We will also mark it as handled to make sure that the default client-side sorting behavior of DataGrid doesn’t kick in.

    private void Customers_Sorting(object sender, DataGridSortingEventArgs e)
    {
        this.ApplySortColumn(e.Column);
        e.Handled = true;
    }

Within ApplySortColumn, we modify the sortDescriptions list to reflect the user’s column clicks. If the column clicked was not in the sortDescriptions list we add it, and if it was we flip its direction. If the user is pressing Shift we keep all existing sortDescriptions, and if he’s not we remove all except the one the user just clicked on.

    private void ApplySortColumn(DataGridColumn column)
    {
        // If column was not sorted, we sort it ascending. If it was already sorted, we flip the sort direction.
        string sortColumn = this.GetColumnSortMemberPath(column);
        CustomSortDescription existingSortDescription = this.sortDescriptions.SingleOrDefault(sd => sd.PropertyName == sortColumn);
        if (existingSortDescription == null)
        {
            existingSortDescription = new CustomSortDescription
            {
                PropertyName = sortColumn,
                Direction = ListSortDirection.Ascending,
                Column = column
            };
            this.sortDescriptions.Add(existingSortDescription);
        }
        else
        {
            existingSortDescription.Direction = (existingSortDescription.Direction == ListSortDirection.Ascending) ? ListSortDirection.Descending : ListSortDirection.Ascending;
        }

        // If user is not pressing Shift, we remove all SortDescriptions except the current one.
        bool isShiftPressed = (Keyboard.Modifiers & ModifierKeys.Shift) != 0;
        if (!isShiftPressed)
        {
            for (int i = this.sortDescriptions.Count – 1; i >= 0; i–)
            {
                CustomSortDescription csd = this.sortDescriptions[i];
                if (csd.PropertyName != sortColumn)
                {
                    this.sortDescriptions.RemoveAt(i);
                }
            }
        }

        this.RefreshData();
    }

The GetColumnSortMemberPath method returns the path of the binding associated with a particular DataGridColumn. The code (shown below) relies on the GetSortMemberPath method in DataGridHelper, which was taken from an earlier blog post by Vincent Sibal. Because each of our actual items is held in a DataWrapper instance (for purposes of data virtualization), all customer properties are accessed through the Data property of DataWrapper. Since the SQL server doesn’t have data wrappers, we don’t want the “Data.” portion of the property path when we build the SQL sorting query; we remove it from the path here.

    private string GetColumnSortMemberPath(DataGridColumn column)
    {
        string prefixToRemove = "Data.";
        string fullSortColumn = DataGridHelper.GetSortMemberPath(column);
        string sortColumn = fullSortColumn.Substring(prefixToRemove.Length);
        return sortColumn;
    }

Notice that after changing the sortDescriptions list in the ApplySortColumn method, we call the RefreshData method. RefreshData gets the portion of the SQL query required for sorting based on the sortDescriptions list, and passes that to the CustomerProvider to be executed on the server. Then it updates the page’s data context to be the newly created CustomerProvider and ensures that the DataGrid columns display the triangles in each column that reflect the sort descriptions.

    private void RefreshData()
    {
        string sortString = this.GetCurrentSortString();
        customerProvider = new CustomerProvider(this.CustomerSinceDatePicker.DateFrom, this.CustomerSinceDatePicker.DateTo, sortString);
        AsyncVirtualizingCollection<Customer> customerList = new AsyncVirtualizingCollection<Customer>(customerProvider, pageSize, timePageInMemory);
        this.DataContext = customerList;

        this.UpdateSortingVisualFeedback();

        this.CustomersDataGrid.SelectedIndex = 0;
    }

As you can see below, GetCurrentSortString enumerates through the sortDescriptions list and creates a string with the SQL syntax that reflects that sorting information. For example, if I click on “First Name” twice, and then shift-click on “Local Calls”, the string returned by this method would be “FirstName DESC, LocalCalls”.

    private string GetCurrentSortString()
    {
        // The result string is created, taking into account all sorted columns in the order they were sorted.
        StringBuilder result = new StringBuilder();
        string separator = String.Empty;
        foreach (CustomSortDescription sd in this.sortDescriptions)
        {
            result.Append(separator);
            result.Append(sd.PropertyName);
            if (sd.Direction == ListSortDirection.Descending)
            {
                result = result.Append(" DESC");
            }
            separator = ", ";
        }

        return result.ToString();
    }

Updating the UI to reflect the sortDescriptions list is very easy:

    private void UpdateSortingVisualFeedback()
    {
        foreach (CustomSortDescription csd in this.sortDescriptions)
        {
            csd.Column.SortDirection = csd.Direction;
        }
    }

Finally, we need to think of the initial sorting state of the app. We would like the customers to be displayed in descending order of their “CustomerSince” dates. To make that happen, we can add the following code to the window’s constructor:

    public MainWindow()
    {
        …
        string defaultSortColumnName = "CustomerSince";
        DataGridColumn defaultSortColumn = this.CustomersDataGrid.Columns.Single(dgc => this.GetColumnSortMemberPath(dgc) == defaultSortColumnName);
        this.sortDescriptions = new List<CustomSortDescription>
        {
            new CustomSortDescription
            {
                PropertyName = defaultSortColumnName,
                Direction = ListSortDirection.Descending,
                Column = defaultSortColumn
            }
        };
        this.RefreshData();
    }

With this code in place, clicking on columns in the DataGrid now causes the corresponding sorting operations to be performed on the server side. This works well in combination with data virtualization. You can see a screenshot of this app below. Notice the triangles in the columns.

If you’re a SQL wizard, I’d love to get your feedback on the stored procedures used in this sample, in particular GetSortedFilteredCustomers. I am by no means a SQL expert and would like to have the current syntax validated or simplified if possible.

Download the database initialization project where you can find the stored procedures (built with .NET 4.0 Beta 2).

Download the data virtualization project with filtering and sorting (built with .NET 4.0 Beta 2).

Posted by Bea under WPF | Comments (4)

November 27, 2009

How can I filter data virtualized items in WPF?

A few months ago, I wrote about a data virtualization solution that combines many of the advantages of two other data virtualization solutions (Paul’s and Vincent’s). Today’s post extends the data virtualization solution in my earlier post by adding the ability to filter the virtualized data. My next post will extend today’s solution further by adding the ability to sort the data, in a way that is seamlessly integrated with the DataGrid UI.

Filtering data virtualized items

WPF offers users the ability to filter data (one way to do that is by using CollectionViewSource, which I discussed in another post in the context of WPF). However, when you filter using the client-side solutions that ship in WPF, all data items need to be accessed. This is fine if you don’t have a lot of items, but for large data sets, it will negate any benefits of virtualization. The trick to get around this problem is to delegate all filtering operations to the server. Today’s sample will show you how to do that.

The scenario for the sample is a list of customers that needs to be filtered by two dates. The list of customers is displayed in a DataGrid, and among its many columns, it displays the date the customer joined a particular mobile phone plan. The user is able to enter two dates, thereby applying a filter that displays only the customers that joined between those two dates.

Run the sample on your machine

Today’s post requires you to have SQL server (or SQL Express) installed. I used VS 2010 Beta 2 to create this project, so you will need to have that installed to open the project. The only new features of WPF 4.0 Beta 2 that are used here were available previously in the toolkit (DataGrid and DatePicker), so you can easily port this project to an older version of VS and .NET if needed. There is no need to have any sample database installed, since we will build our own.

You can download the WPF project that I typically use to create sample databases (for presentations and demo code). Open it in VS 2010, search for the “connectionString” variable in MainWindow.xaml.cs, and replace the name of my sql server instance (SQLEXPRESS) with your own. Then run it and click on the “Initialize Database” button to create the database. This will take a while, so please be patient. When the initialization is complete, you should see “Database has been initialized” displayed in the UI. You can confirm the database’s creation by opening “SQL Server Management Studio” and making sure that the “Customers” database is listed under “Databases”.

You are now ready to open today’s app in VS 2010. Search the solution for “SQLEXPRESS” to find all connection strings and replace them with your own. You should find these in the Customers.dbml, Properties\Settings.settings, and Properties\Settings.Designer.cs pages. You should now be able to press F5 and run the sample.

Data virtualization implementation

In “SQL Server Management Studio”, open the “Customers” database. You will notice that it creates one table – “Customers” – and two stored procedures – “GetCount” and “GetSortedFilteredCustomers” (look under Programmability \ Stored Procedures). “GetCount” returns the number of customers that joined between two specified dates. “GetSortedFilteredCustomers” returns a sorted subset of customers that joined between two specified dates (the subset is defined by specifying the begin and end indices of the desired items). Even though this post won’t focus on sorting, you will see how to pass a simple sorting string to this stored procedure (my next post will cover sorting in depth).

Once the database is in place, the next step is to query it from the client app, which I decided to do using Linq to SQL. If you want to recreate the project, add a new item to the project of type “Linq to SQL classes” and call it “Customers.dbml”. Then open “Server Explorer”, right click “Data connections”, specify your server name, and select the “Customers” database from the drop down. Then expand the data connection until you find the “Customer” table and drag it to the left section of the Linq To SQL’s designer. Similarly, expand the stored procedures section and drag both stored procedures to the right section of the designer.

Once you get to this point, build and inspect the Customers.designer.cs class. You will notice that dragging the Customers table caused a “Customer” class to be generated containing a property for each column in the corresponding table. You will also notice that dragging the stored procedures caused two methods with the same names to be generated. The “GetCount” method has a return type of ISingleResult<GetCountResult> – at the bottom of that file you can see that the GetCountResult class contains one single “Count” property of type int. The “GetSortedFilteredCustomers” method has a return type of int. This stored procedure selects a list of customers, so the return type is not quite right. This is because this stored procedure uses an EXEC to run a SELECT statement created at runtime (more on that in my next post). Replacing the int return type with ISingleResult<Customer> corrects the problem; after that, you need to rebuild the project.

At this point, you are able to execute the stored procedures added to the database from the client app. It’s time to include the data virtualization files from my previous post in this project. You can find the data independent and reusable virtualization files in a separate dll called “DataVirtualization.dll”. The next step is to implement the class that derives from IItemsProvider (which I decided to call CustomerProvider in this project). As a reminder, IItemsProvider requires the implementation of two methods: FetchCount and FetchRange. FetchCount calls the “GetCount” Linq method and stored procedure:

    public int FetchCount()
    {
        CustomersDataContext customersDataContext = new CustomersDataContext();
        count = customersDataContext.GetCount(dateFrom, dateTo).ToList().First().Count.Value;
        return count;
    }

And FetchRange calls the GetSortedFilteredCustomers Linq method and stored procedure:

    public IList<Customer> FetchRange(int startIndex, int pageCount, out int overallCount)
    {
        CustomersDataContext customersDataContext = new CustomersDataContext();

        IList<Customer> customersResult;

        startIndex = startIndex + 1; // SQL index starts at 1.
        int endIndex = startIndex + pageCount – 1; // GetCustomers returns items with indices startIndex through endIndex, inclusive.

        overallCount = count; // In this case it’s ok not to get the count again because we’re assuming the data in the database is not changing.
        customersResult = customersDataContext.GetSortedFilteredCustomers(startIndex, endIndex, this.dateFrom, this.dateTo, this.sortField).ToList();

        return customersResult;
    }

The data virtualization files are all complete. Now we can implement the user interface that uses them.

User interface – DataGrid and DateRangePicker

The first step is to add a DataGrid to MainWindow.xaml. I modified the styles of the actual DataGrid and of each DataGridRow to display feedback to the user when items are being accessed from the database (similar to what I explained in the “IsInitializing + IsLoading” section of my previous post). The XAML for the DataGrid is simple and there is plenty of information online explaining the different properties, so I won’t go into detail about it here.

The next step is to implement the UI for the user to specify the dates to filter by. I decided to group this functionality in a UserControl which I called “DateRangePicker”. In this UserControl, I added two DatePicker controls, one to enter the “from” date and another one to enter the “to” date.

If the user specifies both dates, I have the information I need to find customers with “CustomerSince” date between those dates. I also want to allow finding customers that joined after a certain date or before a certain date, so I need to handle the scenario where only one date is specified. And if the user doesn’t specify any filter dates, all items should be returned.

With these requirements in mind, I decided to expose two properties in DateRangePicker, “DateFrom” and “DateTo”, of type Nullable<DateTime> (null meaning that the customer did not pick a date for that field). Then I two-way data bound these properties and the selected dates of the DatePickers added to the content of this control:

this.DatePickerFrom.SetBinding(DatePicker.SelectedDateProperty, new Binding("DateFrom") { Source = this, Mode = BindingMode.TwoWay });
this.DatePickerTo.SetBinding(DatePicker.SelectedDateProperty, new Binding("DateTo") { Source = this, Mode = BindingMode.TwoWay });

Since my app requires both dates to be in the past, I decided to help the user by preventing selection of dates in the future, which can be done with the following code:

this.DatePickerFrom.BlackoutDates.Add(new CalendarDateRange(DateTime.Today.AddDays(1), DateTime.MaxValue));
this.DatePickerTo.BlackoutDates.Add(new CalendarDateRange(DateTime.Today.AddDays(1), DateTime.MaxValue));

I also decided to expose events that are fired when the “from” and “to” dates are changed:

public event EventHandler FromDateChanged;
public event EventHandler ToDateChanged;

Then I added change handlers for the “from” and “to” properties that I use to fire the events. In addition, if the “from” date is set to be after the “to” date, I clear the “to” date, since this is not a valid state.

    private static void DateFrom_PropertyChanged(DependencyObject obj, DependencyPropertyChangedEventArgs e)
    {
        DateRangePicker dateRangePicker = (DateRangePicker)obj;
        dateRangePicker.DateFromChanged();
    }

    private void DateFromChanged()
    {
        // This updates the blackout dates for DatePickerTo.
        this.DatePickerTo.BlackoutDates.Clear();
        if (this.DateFrom.HasValue)
        {
            DateTime dateFrom = this.DateFrom.Value;
            if (this.DateTo.HasValue)
            {
                DateTime dateTo = this.DateTo.Value;
                if (dateTo <= dateFrom)
                {
                    this.DateTo = null;
                }
            }

            this.DatePickerTo.BlackoutDates.Add(new CalendarDateRange(DateTime.MinValue, dateFrom));
        }
        this.DatePickerTo.BlackoutDates.Add(new CalendarDateRange(DateTime.Today.AddDays(1), DateTime.MaxValue));

        this.OnDateChanged(FromDateChanged);
    }

    private static void DateTo_PropertyChanged(DependencyObject obj, DependencyPropertyChangedEventArgs e)
    {
        DateRangePicker dateRangePicker = (DateRangePicker)obj;
        dateRangePicker.OnDateChanged(dateRangePicker.ToDateChanged);
    }

With this, the code for the DateRangePicker control is complete. Switching back to the main app, I added that control to the page, right above the DataGrid:

<reusableControls:DateRangePicker x:Name="CustomerSinceDatePicker" />

Hooking up the UI to the data

At this point we have all the data virtualization infrastructure, plus an adequate UI for the user to specify the filter parameters and for us to display the customer list. The only thing that’s missing is to hook up the UI to the data.

When the user picks a new date in the DateRangePicker UserControl, we want to re-query the database with the new “from” and “to” date parameters, and display that data in the DataGrid. I did that by adding a handler for the “FromDateChanged” and “ToDateChanged” events of DateRangePicker. Within that handler, I create a new CustomerProvider with the right date parameters, instantiate the data virtualization classes, and re-set the DataContext:

    <reusableControls:DateRangePicker x:Name="CustomerSinceDatePicker" FromDateChanged="CustomerSinceDatePicker_DateChanged" ToDateChanged="CustomerSinceDatePicker_DateChanged"/>

    private void CustomerSinceDatePicker_DateChanged(object sender, EventArgs e)
    {
        this.RefreshData();
    }

    private void RefreshData()
    {
        string sortString = "CustomerSince DESC";
        customerProvider = new CustomerProvider(this.CustomerSinceDatePicker.DateFrom, this.CustomerSinceDatePicker.DateTo, sortString);
        AsyncVirtualizingCollection<Customer> customerList = new AsyncVirtualizingCollection<Customer>(customerProvider, pageSize, timePageInMemory);
        this.DataContext = customerList;

        this.CustomersDataGrid.SelectedIndex = 0;
    }

You may have noticed in the code above that I am passing a sort string to the server – “CustomerSince DESC”. With the current implementation, customers are always displayed in this order. If you have a few pre-defined common sort expressions, you have here all the pieces needed to send those sort descriptions to the server. But what if you want clicking on DataGrid columns to cause sorting in the database? My next post will show how you can do that.

Download the database initialization project (built with .NET 4.0 Beta 2).

Download the data virtualization project with filtering (built with .NET 4.0 Beta 2).

Posted by Bea under WPF | Comments (5)

October 31, 2009

WPF’s CollectionViewSource

CollectionViewSource has existed for a long time in WPF and was recently introduced in Silverlight 3. My next post will cover CollectionViewSource in the context of Silverlight. But before covering that topic, I’ve decided to provide some background about why we introduced this class in WPF.

Views in WPF

When a user binds a WPF property to a collection of data, WPF automatically creates a view to wrap the collection, and binds the property to the view, not the raw collection. This behavior always happens, and is independent of CollectionViewSource.

Views provide four types of functionality: sorting, filtering, grouping and tracking the current item. The scenarios you can implement with these four simple features are endless!

The type of view created by WPF depends on the collection type. There are essentially 3 types of views automatically generated by WPF, all deriving from the CollectionView base class:

ListCollectionView -> Created when the collection implements IList.
BindingListCollectionView -> Created when the collection implements IBindingList.
EnumerableCollectionView -> Created when the collection implements nothing but IEnumerable. This class is internal to WPF.

Before CollectionViewSource was introduced in WPF, you could manipulate the automatically generated view (called the “default view”) by obtaining the ItemCollection returned by the ItemsControl.Items property. ItemCollection is a “hybrid” class – it’s both a collection (implementing ICollection, IList, and IEnumerable) containing the list of items in the ItemsControl, and a view (deriving from CollectionView) that exposes properties to manipulate the default view. Here’s an example:

this.MyItemsControl.Items.SortDescriptions.Add(new SortDescription(…));

ItemCollection doesn’t contain the implementation for any “view” related functionality though – all of its “view” methods are delegated. If the ItemsControl has items added directly to it in XAML (not data bound), ItemCollection delegates view operations to a private property of type “InnerItemCollectionView” (you can find it in Reflector – it’s appropriately called “_internalView”). If the ItemsControl is data bound, the ItemCollection delegates all view operations to the automatically generated default view that wraps the collection.

There is one more view in WPF that you may come across: CompositeCollectionView. Naturally, this view is automatically created when the property is data bound to a CompositeCollection. CompositeCollections have also existed in WPF for a long time. They allow merging several individual collections and individual items into one single collection in XAML.

    <ItemsControl.ItemsSource>
        <CompositeCollection>
            <ListBoxItem><TextBlock Text="Hello" /></ListBoxItem>
            <CollectionContainer Collection="{Binding Source={StaticResource Collection1}}"/>
            <CollectionContainer Collection="{Binding Source={StaticResource Collection2}}"/>
        </CompositeCollection>
    </ItemsControl.ItemsSource>

Instead of using the default view, the user can also create his own view to wrap the collection. This could be a custom view (implementing ICollectionView) or one of the existing public views – typically ListCollectionView or BindingListCollectionView. For example:

    ListCollectionView lcv = new ListCollectionView(myList);
    this.MyItemsControl.ItemsSource = lcv;
    lcv.SortDescriptions.Add(new SortDescription(…));

In this scenario, WPF will not wrap this view in another view, it simply uses the one it’s given, as expected. The user can then use his ListCollectionView object to perform view-related operations (such as adding sort descriptions).

CollectionViewSource in WPF

All of these features already existed in WPF before we decided to add CollectionViewSource. We could already do filtering, sorting, grouping, we could track and set the current item, we could retrieve the default view created by WPF and manipulate it, and we could manually create multiple views of the same collection. So, why did we add CollectionViewSource?

The main reason was to enable those view-related operations in XAML – previously they could only be done in code. Without XAML support, tools like Blend can not provide a good tooling experience for these features. The most common uses of CollectionViewSource are to specify sorting and grouping directly in XAML, or to use XAML to hook up a filter handler defined in code. I’ve shown several samples of this syntax before (this post shows filtering, this post shows grouping, and this post shows sorting and grouping combined), so I won’t go into that here.

CollectionViewSource is NOT a view, unlike the classes I described above. If you look in Reflector, you will notice that it doesn’t even implement ICollectionView – a requirement for a class to be considered a “view”. CollectionViewSource is simply a class that once given a collection (by setting its Source property) creates and exposes the corresponding view (through the View property), and that allows adding sorting and grouping (unfortunately not filtering) directly in XAML.

CollectionViewSource also provides several other methods useful to obtain and manipulate views. Among them is yet another way to retrieve the view that wraps a collection:

ListCollectionView lcv = CollectionViewSource.GetDefaultView(myCollection) as ListCollectionView;
lcv.SortDescriptions.Add(new SortDescription(…));

This is my favorite way of getting a view for a collection because 1) it doesn’t require a handle to the ItemsControl like when using ItemCollection and 2) I don’t need to create the view myself. This method completes the list of ways to retrieve the view for a particular collection.

CollectionViewSource also enables another interesting scenario. If a particular CollectionViewSource points to different collections at different times, it remembers all the views that it created to wrap those collections. If a source that has already been set in the past is set again, CVS recognizes it and reuses the view it created originally. This behavior is useful in hierarchical binding scenarios. To illustrate this point, I created a very simple three-level master-detail scenario with the following XAML:

    <Window.Resources>
        <CollectionViewSource Source="{Binding}" x:Key="cvs1"/>
        <CollectionViewSource Source="{Binding Source={StaticResource cvs1}, Path=Lifts}" x:Key="cvs2"/>
        <CollectionViewSource Source="{Binding Source={StaticResource cvs2}, Path=Runs}" x:Key="cvs3"/>
    </Window.Resources>

The DataContext for this window is set to a data source with the following structure:

    public class Mountains : ObservableCollection<Mountain>
    {
        …
    }

    public class Mountain
    {
        public ObservableCollection<Lift> Lifts { get; set; }
        …
    }

    public class Lift
    {
        public ObservableCollection<string> Runs { get; set; }
        …
    }

Here is a screenshot:

In this sample, cvs1 points to the mountains collection and creates a ListCollectionView to wrap it; cvs2 holds one of three collections: the Lifts collection for the first, second or third mountain; and cvs3 holds one of eight collections of Runs. Now imagine that you pick the first mountain (Whistler) and the second lift from that mountain (Garbanzo Express), then you switch to the second mountain (Stevens Pass) and back to the first mountain again (Whistler). At this point, you would expect the second lift (Garbanzo Express) to still be selected. And it is. Because the second CollectionViewSource stores the view last used to wrap that particular Lifts collection, reusing it next time it needs to display that same collection, the current item is preserved.

If you made it so far, you know more about CollectionViewSource than you will ever need! My next post will include a similar discussion in the context of Silverlight.

Download the WPF project (built with .NET 4.0 Beta 1).

Posted by Bea under WPF | Comments (21)

October 19, 2009

How can I set multiple styles in WPF?

WPF and Silverlight both offer the ability to derive a Style from another Style through the “BasedOn” property. This feature enables developers to organize their styles using a hierarchy similar to class inheritance. Consider the following styles:

With this syntax, a Button that uses RedButtonStyle will have its Foreground property set to Red and its Margin property set to 10.

This feature has been around in WPF for a long time, and it’s new in Silverlight 3.

What if you want to set more than one style on an element? Neither WPF nor Silverlight provide a solution for this problem out of the box. Fortunately there are ways to implement this behavior in WPF, which I will discuss in this blog post.

WPF and Silverlight use markup extensions to provide properties with values that require some logic to obtain. Markup extensions are easily recognizable by the presence of curly brackets surrounding them in XAML. For example, the {Binding} markup extension contains logic to fetch a value from a data source and update it when changes occur; the {StaticResource} markup extension contains logic to grab a value from a resource dictionary based on a key. Fortunately for us, WPF allows users to write their own custom markup extensions. This feature is not yet present in Silverlight, so the solution in this blog is only applicable to WPF.

Others have written great solutions to merge two styles using markup extensions. However, I wanted a solution that provided the ability to merge an unlimited number of styles, which is a little bit trickier.

Writing a markup extension is straightforward. The first step is to create a class that derives from MarkupExtension, and use the MarkupExtensionReturnType attribute to indicate that you intend the value returned from your markup extension to be of type Style.

    [MarkupExtensionReturnType(typeof(Style))]
    public class MultiStyleExtension : MarkupExtension
    {
    }

Specifying inputs to the markup extension

We’d like to give users of our markup extension a simple way to specify the styles to be merged. There are essentially two ways in which the user can specify inputs to a markup extension. The user can set properties or pass parameters to the constructor. Since in this scenario the user needs the ability to specify an unlimited number of styles, my first approach was to create a constructor that takes any number of strings using the “params” keyword:

    public MultiStyleExtension(params string[] inputResourceKeys)
    {
    }

My goal was to be able to write the inputs as follows:

Notice the comma separating the different style keys. Unfortunately, custom markup extensions don’t support an unlimited number of constructor parameters, so this approach results in a compile error. If I knew in advance how many styles I wanted to merge, I could have used the same XAML syntax with a constructor taking the desired number of strings:

    public MultiStyleExtension(string inputResourceKey1, string inputResourceKey2)
    {
    }

As a workaround, I decided to have the constructor parameter take a single string that specifies the style names separated by spaces. The syntax isn’t too bad:

    <Button Style="{local:MultiStyle BigButtonStyle GreenButtonStyle}" … />

    private string[] resourceKeys;

    public MultiStyleExtension(string inputResourceKeys)
    {
        if (inputResourceKeys == null)
        {
            throw new ArgumentNullException("inputResourceKeys");
        }

        this.resourceKeys = inputResourceKeys.Split(new char[] { ‘ ‘ }, StringSplitOptions.RemoveEmptyEntries);

        if (this.resourceKeys.Length == 0)
        {
            throw new ArgumentException("No input resource keys specified.");
        }
    }

Calculating the output of the markup extension

To calculate the output of a markup extension, we need to override a method from MarkupExtension called “ProvideValue”. The value returned from this method will be set in the target of the markup extension.

I started by creating an extension method for Style that knows how to merge two styles. The code for this method is quite simple:

    public static void Merge(this Style style1, Style style2)
    {
        if (style1 == null)
        {
            throw new ArgumentNullException("style1");
        }
        if (style2 == null)
        {
            throw new ArgumentNullException("style2");
        }

        if (style1.TargetType.IsAssignableFrom(style2.TargetType))
        {
            style1.TargetType = style2.TargetType;
        }

        if (style2.BasedOn != null)
        {
            Merge(style1, style2.BasedOn);
        }

        foreach (SetterBase currentSetter in style2.Setters)
        {
            style1.Setters.Add(currentSetter);
        }

        foreach (TriggerBase currentTrigger in style2.Triggers)
        {
            style1.Triggers.Add(currentTrigger);
        }

        // This code is only needed when using DynamicResources.
        foreach (object key in style2.Resources.Keys)
        {
            style1.Resources[key] = style2.Resources[key];
        }
    }

With the logic above, the first style is modified to include all information from the second. If there are conflicts (e.g. both styles have a setter for the same property), the second style wins. Notice that aside from copying styles and triggers, I also took into account the TargetType and BasedOn values as well as any resources the second style may have. For the TargetType of the merged style, I used whichever type is more derived. If the second style has a BasedOn style, I merge its hierarchy of styles recursively. If it has resources, I copy them over to the first style. If those resources are referred to using {StaticResource}, they’re statically resolved before this merge code executes, and therefore it isn’t necessary to move them. I added this code in case we’re using DynamicResources.

The extension method shown above enables the following syntax:

style1.Merge(style2);

This syntax is useful provided that I have instances of both styles within ProvideValue. Well, I don’t. All I get from the constructor is a list of string keys for those styles. If there was support for params in the constructor parameters, I could have used the following syntax to get the actual style instances:

    public MultiStyleExtension(params Style[] styles)
    {
    }

But that doesn’t work. And even if the params limitation didn’t exist, we would probably hit another limitation of markup extensions, where we would have to use property-element syntax instead of attribute syntax to specify the static resources, which is verbose and cumbersome (I explain this bug better in a previous blog post). And even if both those limitations didn’t exist, I would still rather write the list of styles using just their names – it is shorter and simpler to read than a StaticResource for each one.

The solution is to create a StaticResourceExtension using code. Given a style key of type string and a service provider, I can use StaticResourceExtension to retrieve the actual style instance. Here is the syntax:

Style currentStyle = new StaticResourceExtension(currentResourceKey).ProvideValue(serviceProvider) as Style;

Now we have all the pieces needed to write the ProvideValue method:

    public override object ProvideValue(IServiceProvider serviceProvider)
    {
        Style resultStyle = new Style();

        foreach (string currentResourceKey in resourceKeys)
        {
            Style currentStyle = new StaticResourceExtension(currentResourceKey).ProvideValue(serviceProvider) as Style;

            if (currentStyle == null)
            {
                throw new InvalidOperationException("Could not find style with resource key " + currentResourceKey + ".");
            }

            resultStyle.Merge(currentStyle);
        }
        return resultStyle;
    }

Here is a complete example of the usage of the MultiStyle markup extension:

    <Window.Resources>
        <Style TargetType="Button" x:Key="SmallButtonStyle">
            <Setter Property="Width" Value="120" />
            <Setter Property="Height" Value="25" />
            <Setter Property="FontSize" Value="12" />
        </Style>

        <Style TargetType="Button" x:Key="GreenButtonStyle">
            <Setter Property="Foreground" Value="Green" />
        </Style>

        <Style TargetType="Button" x:Key="BoldButtonStyle">
            <Setter Property="FontWeight" Value="Bold" />
        </Style>
    </Window.Resources>

    <Button Style="{local:MultiStyle SmallButtonStyle GreenButtonStyle BoldButtonStyle}" Content="Small, green, bold" />

Download the WPF project (built with .NET 4.0 Beta 1).

Posted by Bea under WPF | Comments (9)

October 5, 2009

How can I improve on existing WPF data virtualization solutions?

In a previous post, I compared two data virtualization techniques implemented by Paul McClean and Vincent Van Den Berghe for WPF. In this post, I describe a solution that combines some of the best features of both. I started with Paul’s solution, eliminated a few limitations, and incorporated some of Vincent’s ideas.

Selection

In Paul’s solution, a “collection reset” event is used to notify the UI each time a new page is loaded from the database. As a side effect, this notification unintentionally causes a ListBox to lose track of the selected item. This makes it impossible for a user to scroll through a long list using the down-arrow key; every time a new page is loaded, the ListBox selection jumps back to the beginning of the list. The troublesome code can be found in the following methods of AsyncVirtualizingCollection:

    private void LoadPageCompleted(object args)
    {
        int pageIndex = (int)((object[]) args)[0];
        IList<T> page = (IList<T>)((object[])args)[1];

        PopulatePage(pageIndex, page);
        IsLoading = false;
        FireCollectionReset();
    }

    private void FireCollectionReset()
    {
        NotifyCollectionChangedEventArgs e = new NotifyCollectionChangedEventArgs(NotifyCollectionChangedAction.Reset);
        OnCollectionChanged(e);
    }

One possible solution to this problem is to provide more fine-grained add and remove notifications for the new items, instead of a collection reset. Implementing this is not as straightforward as it seems, though, because of the combination of the following two behaviors : 1) When WPF receives a collection change notification for a newly added item, ListCollectionView accesses that item using the collection’s indexer, even if the item is not visible in the UI. 2) When an item is accessed, Paul’s caching heuristics load its page into memory, as well as the previous or next page depending on whether the item belongs to the first of second half of its page.

With this information, you can probably guess what happens when we provide fine-grained collection notifications. When a page is loaded, we notify WPF that a few items were added to the collection, ListCollectionView accesses each one of those items one by one, triggering a load of the subsequent page, which in turn notifies WPF that a more items were added to the collection, which causes the ListCollectionView to access each one, and so on. Eventually, the whole collection gets loaded, which is exactly what we’re trying to avoid.

We could use fine-grained notifications with either of two possible approaches: 1) change the caching heuristics so that neighboring pages are no longer loaded; or 2) implement our own view (as a replacement for ListCollectionView) that doesn’t call the collection indexer to access each newly added item. Either approach would fix the problem, but they would not fix another related problem. If we happen to select an item that is not yet loaded, selection would be lost when the item finishes loading. This would happen because selection is tracked based on the actual data item – not its index within the ListBox. If I press the down-arrow key until I select an item that hasn’t yet been loaded, when its data item changes at load time (from null to the actual data), the ListBox’s selected item is no longer referring to that same item.

This train of thought made it clear that Vincent’s technique of wrapping each data item could solve all these selection issues. When using data wrappers, the data items associated with each ListBoxItem don’t ever change – they’re the wrappers themselves. The data wrappers are not replaced when data loads, and therefore WPF doesn’t lose track of the selected item. What changes is the data within the wrapper, which means we can now raise property change notifications to update the UI, instead of collection change notifications. This is good news, since property change notifications are very fine-grained, and they work across threads.

My data wrapper class is called DataWrapper and among other properties it contains a reference to the actual data.

    public class DataWrapper<T> : INotifyPropertyChanged where T : class
    {
        private T data;
        …
        public T Data
        {
            get { return this.data; }
            internal set
            {
                this.data = value;
                this.OnPropertyChanged("Data");
                …
            }
        }
        …
    }

Adding wrappers required some changes in the collection code base. In Paul’s code, requesting a page would add a new entry in the page dictionary with value null, and populating a page would set that value to the actual page:

    protected virtual void RequestPage(int pageIndex)
    {
        if (!_pages.ContainsKey(pageIndex))
        {
            _pages.Add(pageIndex, null);
            _pageTouchTimes.Add(pageIndex, DateTime.Now);
            LoadPage(pageIndex);
        }
        else
        {
            _pageTouchTimes[pageIndex] = DateTime.Now;
        }
    }

    protected virtual void PopulatePage(int pageIndex, IList<T> page)
    {
        if ( _pages.ContainsKey(pageIndex) )
            _pages[pageIndex] = page;
    }

To support data wrappers, I changed the code so that a request for a new page results in the immediate creation of a page full of empty data wrappers . This page is added to the dictionary right away. Later, when the actual data gets loaded, populating the page just fills in the data part of the wrappers.

    protected virtual void RequestPage(int pageIndex)
    {
        if (!_pages.ContainsKey(pageIndex))
        {
            int pageLength = Math.Min(this.PageSize, this.Count – pageIndex * this.PageSize);
            DataPage<T> page = new DataPage<T>(pageIndex * this.PageSize, pageLength);
            _pages.Add(pageIndex, page);
            LoadPage(pageIndex, pageLength);
        }
        else
        {
            _pages[pageIndex].TouchTime = DateTime.Now;
        }
    }

    protected virtual void PopulatePage(int pageIndex, IList<T> dataItems)
    {
        DataPage<T> page;
        if (_pages.TryGetValue(pageIndex, out page))
        {
            page.Populate(dataItems);
        }
    }

Contains and IndexOf

In Paul’s data virtualization solution, VirtualizingCollection does not include an implementation for the Contains and IndexOf methods:

    public bool Contains(T item)
    {
        return false;
    }

    public int IndexOf(T item)
    {
        return -1;
    }

As a result, the CurrentItem property of WPF’s collection view doesn’t track the current item correctly, and therefore we can’t implement the Master-Detail scenario by simply binding both a ListBox and a ContentControl to the collection. There are other scenarios equally affected by this.

Providing an implementation for these methods was relatively straightforward:

    public bool Contains(DataWrapper<T> item)
    {
        foreach (DataPage<T> page in _pages.Values)
        {
            if (page.Items.Contains(item))
            {
                return true;
            }
        }
        return false;
    }

    public int IndexOf(DataWrapper<T> item)
    {
        foreach (KeyValuePair<int, DataPage<T>> keyValuePair in _pages)
        {
            int indexWithinPage = keyValuePair.Value.Items.IndexOf(item);
            if (indexWithinPage != -1)
            {
                return PageSize * keyValuePair.Key + indexWithinPage;
            }
        }
        return -1;
    }

Currency

Providing an implementation for Contains and IndexOf enabled currency (CurrentItem), but there were still some corner cases that didn’t work correctly. For example, if I selected an item and then scrolled it off-screen, WPF knew not to virtualize the UI element for that item, but the data was still being virtualized. This also caused problems with currency.

I needed a way to prevent an item from virtualizing its data if its UI was still available. Adding data wrappers had the fortunate side effect of making the fix for this problem easier. I know that a data wrapper is being used if someone is listening to its property change event. So I was able to add an IsInUse property to the data wrapper with the following implementation:

    public class DataWrapper<T> : INotifyPropertyChanged where T : class
    {
        …
        public event PropertyChangedEventHandler PropertyChanged;
        public bool IsInUse
        {
            get { return this.PropertyChanged != null; }
        }
    }

Similarly, I added a property that determines whether a page has at least one item in use:

    public class DataPage<T> where T : class
    {
        …
        public bool IsInUse
        {
            get { return this.Items.Any(wrapper => wrapper.IsInUse); }
        }
    }

Then I used that property to avoid cleaning up pages that are still in use, within VirtualizingCollection:

    public void CleanUpPages()
    {
        int[] keys = _pages.Keys.ToArray();
        foreach (int key in keys)
        {
            // page 0 is a special case, since WPF ItemsControl access the first item frequently
            if (key != 0 && (DateTime.Now – _pages[key].TouchTime).TotalMilliseconds > PageTimeout)
            {
                bool removePage = true;
                DataPage<T> page;
                if (_pages.TryGetValue(key, out page))
                {
                    removePage = !page.IsInUse;
                }

                if (removePage)
                {
                    _pages.Remove(key);
                }
            }
        }
    }

IsInitializing + IsLoading

Paul’s AsyncVirtualizingCollection has an “IsLoading” property that is set to true when the collection is either counting its items or fetching a page. This is useful so that we can provide visual feedback when we’re querying data from the database. On the other hand, it’s a bit limiting to have only one property indicating that work is in progress. We don’t want to prevent the user from interacting with other items in the ListBox just because scrolling causes a few items to start downloading. Ideally, we would get more fine-grained status information.

To solve this problem, I added an “IsInitializing” property that is true when we’re fetching the count, and changed “IsLoading” slightly to inform us when the collection is fetching a new page. The “IsInitializing” property is defined at the collection level, and the “IsLoading” property is defined in the data wrapper.

When the collection count is being fetched (that is, when IsInitializing is true), I display a message in the middle of the empty ListBox and switch to the “Wait” cursor, making it obvious that it’s not yet ready for user interaction:

When an item is being fetched from the database (that is, when IsLoading is true), I display a message and “Wait” cursor just within the corresponding ListViewItem:

This is the point where I would normally hand the problem over to a visual designer or an interaction designer. Now that we can get fine-grained information about which data items are loading and which are available, a designer could come up with a variety of ways to display this information to the user.

Still missing…

Paul’s solution assumes the collection is read-only, and my code doesn’t really fix that limitation. Although my AsyncVirtualizingCollection will notice if its count has changed when fetching a new page of data, it won’t notice at any other time. If you’re successful at extending this solution to support dynamic collection changes, I’d love to hear from you!

You can download the source for this project.

Posted by Bea under WPF | Comments (26)

September 7, 2009

How can I add labels to a WPF pie chart? – Implementation details

Update April 4 2010: The LabeledPieChart code in this post has been updated to the latest WPF and Silverlight toolkits. You can find more details in this blog post.

In my last blog post, I showed how you can use a custom control to add labels to a WPF pie chart. In this post I will discuss its implementation.

I started by thinking about what I wanted the usage syntax to look like. Ideally, I would like to create a LabeledPieSeries class that derives from PieSeries, and that exposes a LabelStyle property. This would make using a pie series with labels as easy as using any other chart series, which WPF chart users are already familiar with (the Codeplex charting overview shows what that syntax would look like). Unfortunately, PieSeries is currently sealed, so I had to search for another alternative.

Often, when faced with this design constraint, using a behavior is a good alternative. A behavior is simply an attached property that provides access to the element it’s attached to, and therefore to the whole tree it belongs to. I used a similar technique in my drag and drop blog post. More recently, Blend 3 has embraced this technique by providing support for applying behaviors at design time, and has generated a community around the creation of reusable behaviors. You can read more about Blend’s support for behaviors in Christian’s blog.

Behavior

The implementation of this behavior consists of an attached DependencyProperty “IsLabeled,” defined on a static class called PieLabelBehavior. This DP is intended to be attached to a PieDataPoint. When the DP is attached, the behavior creates a new label (instantiates a PieChartLabel), sets many of its properties, and walks the tree to add it to the desired location. Attaching the DP to the corresponding PieDataPoint provides an easy and reliable way to be notified when the PieDataPoints are created and added to the tree, and therefore ready to have the corresponding labels generated. Attaching the DP to PieSeries instead would have simplified setting the DP, since there would only be one place to set it, but it would have made it harder to know exactly when the PieDataPoints are added to the tree. Here is how the behavior is attached to each PieDataPoint:

To avoid having neighboring wedges overlap a pie label, all labels had to be positioned in their own Canvas, which has a higher z-order than the PieDataPoint’s Canvas. My first attempt was to add the label’s Canvas to the PieSeries’ template. I added a Grid which had as its children the already existing PlotArea Canvas and my newly created Canvas that would contain the labels. The problem with this approach was that PieSeries clips itself, causing longer labels to be cut off. I attempted to have the actual pie chart occupy a smaller percentage of the PieSeries’ area, but unfortunately charting source code is setting the chart to take 95% of PieSerie’s space. Since this is percentage is not configurable, I had to find another solution. So, instead, I moved the label’s canvas to the Chart template. This fixed the clipping issue.

At this point, I have a way to inject code in a PieDataPoint when it’s added to the tree and I have a location to add my labels. The missing piece is the easiest part: when the PieDataPoint is added to the tree, I walk up the visual tree to find the label area Canvas, create a PieChartLabel and add it to the Canvas. This was done in the change handler for the IsLabeled attached DP (this handler was attached during the DP’s registration, as part of its PropertyMetadata):

    private static void IsLabeledPropertyChanged(DependencyObject obj, DependencyPropertyChangedEventArgs e)
    {
        bool isLabeled = (bool)e.NewValue;
        if (isLabeled == true)
        {
            PieDataPoint pieDataPoint = obj as PieDataPoint;
            if (pieDataPoint != null)
            {
                Chart chart = TreeHelper.FindAncestor<Chart>(pieDataPoint.Parent as DependencyObject);
                if (chart != null)
                {
                    Canvas labelArea = chart.Template.FindName("LabelArea_PART", chart) as Canvas;
                    if (labelArea != null)
                    {
                        AddLabel(pieDataPoint, labelArea);
                    }
                }
            }
        }
    }

The AddLabel method creates a new PieChartLabel, sets its properties – many of them reflecting information found in the corresponding PieDataPoint, which we have a handle to – and adds it as a child of the label area Canvas passed as a parameter. AddLabel also contains code that removes the label when the PieDataPoint is unloaded:

    pieDataPoint.Unloaded += delegate
    {
        labelArea.Children.Remove(label);
    };

The next step was to figure out how the labels were going to be positioned in the Canvas. I decided to have four “display modes”, which I explain in my previous blog post. Picking the desired display mode can be done through a DisplayMode property on PieChartLabel. Below I discuss the implementation of each mode.

ArcMidpoint mode

In order to implement this mode, I first needed to calculate the midpoint of the PieDataPoint’s arc. To get that information, I created a PieChartHelper static class containing a GetPieChartInfo method with the following signature:

public static bool GetPieChartInfo(Geometry geometry, out Point center, out Point arcMidpoint, out bool isArcSmall)

The “geometry” parameter is the wedge geometry of the corresponding PieDataPoint. This method returns three values: “center” which we’ll use in the Connected mode, “arcMidPoint” which is the midpoint of the arc needed for this mode, and “isArcSmall” which we’ll use in the AutoMixed mode. More on the Connected and AutoMixed modes later.

I won’t go into the implementation details for this method here. It requires a bit of understanding of math, and very little of WPF. If there’s interest, I’ll write a separate blog post for it – leave a comment or send me email if this interests you.

PieDataPoint exposes its Geometry through a DP, which is handy in this scenario. When I create a new PieChartLabel (in the behavior), I data bind PieChartLabel’s Geometry DP to the PieDataPoint’s Geometry. This ensures that a change in the PieDataPoint’s Geometry causes the PieChartLabel’s Geometry to change too. With this in mind, the change handler for PieChartLabel’s Geometry is a good location to call GetPieChartInfo:

    private Point center;
    private Point arcMidpoint;
    public bool IsArcSmall { get; set; }
    …

    private static void GeometryPropertyChanged(DependencyObject obj, DependencyPropertyChangedEventArgs e)
    {
        PieChartLabel label = obj as PieChartLabel;
        if (label != null)
        {
            bool isArcSmall;
            PieChartHelper.GetPieChartInfo(e.NewValue as Geometry, out label.center, out label.arcMidpoint, out isArcSmall);
            label.IsArcSmall = isArcSmall;
        }
    }

Now that I have the arc midpoint, the next step is to position the label where I want it when the DisplayMode is set to “ArcMidpoint”. In the registration for the DisplayMode property, I specify that setting this property affects arrange, and ensured that the labels get repositioned during arrange. In the explanation for Auto mode it will become clear why I decided to reposition labels on arrange, instead of simply calling the PositionLabel method from within the DisplayMode’s change handler.

    protected override Size ArrangeOverride(Size arrangeBounds)
    {
        this.PositionLabel();
        return base.ArrangeOverride(arrangeBounds);
    }

    private void PositionLabel()
    {
        switch (this.DisplayMode)
        {
            case DisplayMode.ArcMidpoint:
                this.PositionArcMidpoint();
                break;
            case DisplayMode.Connected:
                this.PositionConnected();
                break;
            case DisplayMode.AutoMixed:
                this.PositionAutoMixed();
                break;
            case DisplayMode.Auto:
                this.PositionAuto();
                break;
        }
    }

With this code in place, the method that actually centers the label on the arc midpoint is very simple:

    private void PositionArcMidpoint()
    {
        this.RemovePolyline();

        if (this.contentPart != null)
        {
            Canvas.SetTop(this.contentPart, this.arcMidpoint.Y – 0.5 * this.contentPart.DesiredSize.Height);
            Canvas.SetLeft(this.contentPart, this.arcMidpoint.X – 0.5 * this.contentPart.DesiredSize.Width);
        }
    }

I will explain in the next section why I call RemovePolyline() at the beginning of this method.

Connected mode

To create the line that connects the wedge to the label in this mode, we have to obtain the coordinates for each of its three points. We already have the first point – it’s the arc midpoint.

The second point should be placed on the line defined by the center of the chart and the arc midpoint. This is why I am returning the center of the chart from the GetPieChartInfo method. I decided that the second point would be 10 pixels away from the arc midpoint, in the outward direction. You may want to change this value or provide a DP to make it configurable.

The third point is placed 20 pixels away from the second point, on the X axis. If the wedge is on the right side of the chart the point is placed towards the right, otherwise it’s placed towards the left.

    private void PositionConnected()
    {
        this.RemovePolyline();

        if (this.contentPart != null)
        {
            PointCollection newPoints = new PointCollection();

            // First point
            newPoints.Add(this.SnapPoint(this.arcMidpoint));

            // Second point
            Vector radialDirection = this.arcMidpoint – this.center;
            radialDirection.Normalize();
            Point secondPoint = this.arcMidpoint + (radialDirection * 10);
            newPoints.Add(this.SnapPoint(secondPoint));

            // Third point
            int sign = Math.Sign(radialDirection.X); // 1 if label is on the right side, -1 if it’s on the left.
            Point thirdPoint = secondPoint + new Vector(sign * 20, 0);
            newPoints.Add(this.SnapPoint(thirdPoint));

            double contentX = (sign == 1) ? thirdPoint.X : thirdPoint.X – this.contentPart.DesiredSize.Width;
            double contentY = thirdPoint.Y – 0.5 * this.contentPart.DesiredSize.Height;
            Canvas.SetTop(this.contentPart, contentY);
            Canvas.SetLeft(this.contentPart, contentX);

            Polyline polyline = new Polyline();
            polyline.Points = newPoints;
            polyline.SetBinding(Polyline.StrokeThicknessProperty, new Binding("LineStrokeThickness") { Source = this });
            polyline.SetBinding(Polyline.StrokeProperty, new Binding("LineStroke") { Source = this });
            polyline.StrokeLineJoin = PenLineJoin.Round;

            this.canvasPart.Children.Add(polyline);
        }
    }

You may be wondering why I am creating a new Polyline every time the line needs to be updated. It would be more efficient to add a Polyline to the PieChartLabel’s ControlTemplate, and simply replace its points in this method. Unfortunately, a WPF rendering bug prevents that solution from working. This is why we need to call the RemovePolyline() method every time we’re about to reposition a label.

AutoMixed mode

The AutoMixed mode uses the size of the arc to decide how to position the label: if the arc is small, it uses the Connected mode, otherwise it uses the ArcMidpoint mode. The GetPieChartInfo helper method returns an “isArcSmall” boolean value indicating which mode to use:

    private void PositionAutoMixed()
    {
        if (this.IsArcSmall)
        {
            this.PositionConnected();
        }
        else
        {
            this.PositionArcMidpoint();
        }
    }

Auto mode

In the Auto mode, if at least one wedge is small, all labels are positioned using the Connected mode. Otherwise, they are all positioned using the ArcMidpoint mode. This behavior is tricky to implement. The positioning mode decision can not be made at the label level because it requires knowledge of the arc size of all other PieDataPoints.

The canvas containing all the labels is a perfect place to decide how each label should be positioned because it knows about all labels. So, I replaced that canvas with a custom control deriving from Canvas which I called PieChartLabelArea. Then I added a property called HasSmallArc which is true if at least one wedge is small. This DP is set in the MeasureOverride method:

    protected override Size MeasureOverride(Size constraint)
    {
        bool hasSmallArc = false;
        foreach (PieChartLabel label in this.Children.OfType<PieChartLabel>())
        {
            if (label.IsArcSmall)
            {
                hasSmallArc = true;
                break;
            }
        }
        this.HasSmallArc = hasSmallArc;

        return base.MeasureOverride(constraint);
    }

When HasSmallArc changes, its change handler is called, which invalidates arrange for all labels.

    private static void HasSmallArcPropertyChanged(DependencyObject obj, DependencyPropertyChangedEventArgs e)
    {
        PieChartLabelArea labelArea = obj as PieChartLabelArea;
        if (labelArea != null)
        {
            foreach (PieChartLabel label in labelArea.Children.OfType<PieChartLabel>())
            {
                label.InvalidateArrange();
            }
        }
    }

We’ve already seen previously that PieChartLabel’s ArrangeOverride re-positions the label.

Let’s run through a common execution example. Imagine that during MeasureOverride the PieChartLabelArea control discovers that one of its labels has a small arc. The label area’s IsSmallArc is set to true, which causes its change handler to execute, which invalidates arrange on the labels, which causes all labels to be re-positioned using the Auto mode. At this point, label area’s HasSmallArc is set to true, and all labels can walk the tree to get to it. With this in mind, the following code of PieChartLabel should be easy to understand:

    private void PositionAuto()
    {
        Chart chart = TreeHelper.FindAncestor<Chart>(this);
        if (chart != null)
        {
            PieChartLabelArea labelArea = chart.Template.FindName("LabelArea_PART", chart) as PieChartLabelArea;
            if (labelArea != null && labelArea.HasSmallArc)
            {
                this.PositionConnected();
            }
            else
            {
                this.PositionArcMidpoint();
            }
        }
    }

The style for the Chart is identical to the default style, but with the PieChartLabelArea control added to it, on top of all other elements:

The chart custom control

Most of the label functionality is working at this point, but I still don’t have a way to specify the Style for the labels. Ideally, I would derive from the Chart control, add a LabelStyle DP to my custom chart control, and provide the style above (with the PieChartLabelArea) as its default style. Unfortunately, Chart is also sealed, so I had to think of other alternatives.

I could have added another attached property containing the LabelStyle, to be applied to the Chart control. If I had done this, usage of a pie chart with labels would require: 1) creating a chart with a PieSeries in XAML as usual, 2) setting the Chart’s style to the Style that contains PieChartLabelArea, 3) setting PieSeries’ StylePalette to the style that attaches IsLabeled to each PieDataPoint (I show this Style in the Behavior section), and 4) setting the LabelStyle attached DP on the Chart itself. I wasn’t happy with the complexity of this option – I wanted to shield the developer from some of this complexity.

Instead, I opted to create a custom control (derived from Control) containing the Chart and PieSeries in its template. I called it LabeledPieChart:

LabeledPieChart exposes a property called PieChartLabelStyle where the developer can specify the look for all labels. For added convenience, it also exposes a PieChartLabelItemTemplate DP to specify the DataTemplate. If these are not set I provide a reasonable default look.

The upside of implementing this custom control is that the developer doesn’t have to worry about attaching DPs and setting Styles to use a labeled pie chart. The downside is that all properties in Chart and PieSeries that would allow the developer to customize the chart are now hidden. One solution is to expose all of those properties in LabeledPieChart and template bind them to the ones in Chart and PieSeries. I exposed a few properties that I found most useful, as an example. As you use this control you may need to expose other properties.

Selection

Last, I wanted to enable selection for the PieChart, and I wanted to make clicking a label have the same effect as clicking the associated PieDataPoint.

I started by exposing the IsSelectionEnabled property of PieSeries in my LabeledPieChart control. Enabling selection is as easy as setting this DP on the control:

<customControls:LabeledPieChart IsSelectionEnabled="True" … />

I also had to expose the SelectedItem property, but with a slightly different behavior. For this property, I want changes in the LabeledPieChart’s SelectedItem to be reflected in the PieSeries’ SelectedItem and vice versa. If this property is set by the user the UI needs to reflect it, and if it’s set by internal code it needs to return the right value to the user. This is why I use a two-way binding to the templated parent, as you can see in the LabeledPieChart’s default Style above. With this in place, I only had to make sure that clicking on the label would change SelectedItem, which was done with the following code in PieChartLabel:

    protected override void OnMouseLeftButtonDown(MouseButtonEventArgs e)
    {
        Chart chart = TreeHelper.FindAncestor<Chart>(this);
        if (chart != null)
        {
            PieSeries pieSeries = chart.Series.OfType<PieSeries>().FirstOrDefault();
            if (pieSeries != null)
            {
                pieSeries.SelectedItem = this.Content;
            }
        }
    }

Summary

By helping you understand this implementation of pie charts labels, my goal is to inspire you to extend this code and adapt it to your particular scenario. And hopefully you learned some new tricks that will help you in other scenarios with similar technical problems.

After I wrote my last blog post, I fixed a couple of bugs in the code, so if you’re using the old code make sure you download it again.

In my next blog post, I will talk about the experience of porting this code to Silverlight.

You can see this sample running as an xbap or download its source.

Posted by Bea under WPF | Comments (7)

August 23, 2009

How can I add labels to a WPF pie chart?

Update April 4 2010: The LabeledPieChart code in this post has been updated to the latest WPF and Silverlight toolkits. You can find more details in this blog post.

The WPF Toolkit and Silverlight Toolkit both include a very versatile chart control. Although support for labels (or annotations) is a frequently requested feature, it is not yet present in the current toolkits. David Anson blogged about a great solution to add labels to a ColumnSeries. In the next few posts, I will talk about one way to add labels to a PieSeries.

In this post, I will show how you can use my custom control that provides annotations for the WPF pie chart; in the next post, I will discuss the control’s implementation; and in the post after that I will show you the process of converting the WPF code to Silverlight.

Basic usage

You can use the following XAML to display a labeled pie chart with your data without any further customizations.

    <customControls:LabeledPieChart
        Title="Population of Puget Sound Cities"
        ItemsSource="{Binding}"
        IndependentValuePath="Name"
        DependentValuePath="Population" />

This markup assumes that the data context is set to the following ObservableCollection:

    this.cities = new ObservableCollection<City>
    {
        new City { Name = "Bellevue", Population = 121347 },
        new City { Name = "Issaquah", Population = 23363 },
        new City { Name = "Redmond", Population = 49427 },
        new City { Name = "Seattle", Population = 602000 },
        new City { Name = "Kirkland", Population = 47325 }
    };
    this.DataContext = this.cities;

Here is the result of the XAML and code above:

Label display modes

The LabeledPieChart control has four ways it can position each label:

ArcMidpoint: The label is centered on the midpoint of the circular arc of the corresponding pie wedge.
Connected: The label is positioned outside the pie chart, with a short line connecting it to its pie wedge.
AutoMixed: Small pie wedges display their labels using the connected mode, and bigger wedges use the arc midpoint mode.
Auto: If at least one pie wedge is small, all wedges use the connected mode. If all wedges are big, they all use the arc midpoint mode. Dynamically adding and removing wedges may cause all wedges to alternate between arc midpoint and connected mode.

The images below illustrate the label modes (except Auto, which looks like ArcMidpoint or Connected).

ArcMidpoint	Connected	AutoMixed

To switch between these modes, you can set the LabelDisplayMode property of the LabeledPieChart control:

    <customControls:LabeledPieChart
        LabelDisplayMode="Auto"
        … />

Customizing the content of the labels

To configure the content of the labels, you can define a DataTemplate that displays the data you’re interested in, using the PieChartLabelItemTemplate property of LabeledPieChart:

The content for PieChartLabel – the custom control that displays a single label – is the actual City data item. Therefore, if you want to display properties of the data item within the label, you can bind to them by just specifying the Path.

Notice that I am using an AncestorType binding to get at the FormattedRatio property. FormattedRatio is a very handy property that gives you the percentage of the numeric value represented in the current wedge, in relation to the sum of all items. This property is defined originally on PieDataPoint, and also on PieChartLabel. PieChartLabel’s FormattedRatio is data bound to the one in PieDataPoint, so they’re kept in sync. Since the DataTemplate’s templated parent is a ContentPresenter, using TemplateBinding would try to find the FormattedRatio property on the ContentPresenter itself, which would of course fail. So, I need to keep walking up the tree to find the PieChartLabel control so that I can bind to that property. AncestorType provides an easy way of doing that.

Customizing the connector line

The connecting line’s thickness and brush are exposed as properties in PieChartLabel, and therefore can be customized easily in its style. Here’s an example:

Master-detail scenario

To implement the master-detail scenario using this chart, you simply need to set the IsSelectionEnabled property exposed by LabeledPieChart, and bind another element to the chart’s selected item:

Clicking on the label has the same effect as clicking on the PieDataPoint associated with it – it changes the selection of the details section.

Summary

This blog post provided a quick overview of LabeledPieChart control’s main features. Hopefully these features cover many scenarios people are looking for. However, my guess is that this code will be used as a starting point for customized scenarios. My next post discusses some implementation details, which I am hoping will inspire developers to customize and extend the control for their own needs. So stay tuned.

For more information about Charting, I recommend subscribing to David Anson’s blog – there is no better charting resource out there!

You can see this sample running as an xbap or download its source.

Posted by Bea under WPF | Comments (11)

July 26, 2009

Data virtualization

My last post covered the current UI virtualization support on Silverlight and WPF. In this post, I will discuss another technique that helps improve performance when dealing with large data sets: data virtualization.

A control that uses data virtualization knows to keep in memory no more data items than the ones needed for display. For example, if a data-virtualized ListBox is data bound to 1,000,000 data items and only displays 100 at a single point in time, it only keeps about 100 data items in memory. If the user scrolls, the data-virtualized ListBox knows to swap the items in memory in a way that optimizes performance and memory consumption.

I talked about a possible solution that provides partial data virtualization for TreeView in a previous post. I discussed that solution in the context of WPF, but it could just as easily be used for Silverlight. The discussion in this post addresses data virtualization for non-hierarchical scenarios.

Data virtualization in WPF

WPF has no built-in generic support for data virtualization.

Fortunately, Paul McClean and Vincent Van Den Berghe have recently come up with approaches to work around this limitation (the link to Vincent’s code in the pdf is incorrect, so I’ve uploaded his code here). These two solutions haven’t been advertised enough, in my opinion! They’re both great solutions and easy to adapt to your specific scenario.

Paul and Vincent’s solutions are very similar to each other – they both implement a layer that is capable of managing data items that are kept in memory. This layer knows when to fetch more data items and when to let old data items be garbage collected, based on the user’s scrolling patterns.

Both solutions take advantage of the fact that when a WPF ListBox is data bound to an IList, it only accesses the list’s Count property and the data items it needs to display on the screen. Both Paul and Vincent implemented a “fake” collection that knows the overall count of the real collection and knows how to retrieve a particular data item, even if that data item is not in memory. This collection essentially manages a cache of items and, when asked for new data items, it’s capable of retrieving a new set of data items from the database, discarding data items that are no longer in use.

The implementation details are quite different, though (this is expected – neither of them was aware of the other one’s work).

Paul implemented his own collection called VirtualizingCollection<T>. This collection has a handle to an IItemsProvider, which is a class that knows how to fetch the overall item count and a consecutive list of items from the database (or web service, XML file or whatever data store you’re using). VirtualizingCollection<T> keeps the list of non-virtualized data items and decides when new items should be retrieved. Its indexer immediately returns a data item if it’s already in memory, or loads more data from the database if it’s not. Paul also implemented an extension to this collection called AsyncVirtualizingCollection<T> that provides the ability to do expensive query operations without blocking the UI thread.

Vincent implemented not only his own collection, but also a custom view type for that collection. His collection (VirtualList<T>) implements ICollectionViewFactory, which allows the collection to say that it wants WPF to create a VirtualListCollectionView<T> when binding to it. His view has a handle to the collection and knows to forward any data loading operations to the collection, as well as any sorting and filtering the user may have specified. The data item caching code is in VirtualListBase<T>, which is the base class for both the view and the collection. Vincent also implements a DataRefBase<T> class that wraps each data item, exposing all of its properties using the ICustomTypeDescriptor interface, and adding property change notifications if not present in the original data type.

At the end of this post, I link to an xbap that shows both solutions running side-by-side. Below I compare different aspects of these solutions, and discuss pros and cons of each solution when applicable.

Using these solutions in your project

If you want to use Paul’s solution, you need to provide your own implementation of IItemsProvider. This is where you would add code that retrieves the overall count and a page of items from your database. Then you just need to bind your control to a collection of type AsyncVirtualizingCollection<T>. Here is the example included in Paul’s sample code:

    public class DemoCustomerProvider : IItemsProvider<PaulsCustomer>
    {
        private readonly int _count;
        private readonly int _fetchDelay;

        public DemoCustomerProvider(int count, int fetchDelay)
        {
            _count = count;
            _fetchDelay = fetchDelay;
        }

        public int FetchCount()
        {
            Thread.Sleep(_fetchDelay);
            return _count;
        }

        public IList<PaulsCustomer> FetchRange(int startIndex, int count)
        {
            Thread.Sleep(_fetchDelay);

            List<PaulsCustomer> list = new List<PaulsCustomer>();
            for (int i = startIndex; i < startIndex + count; i++)
            {
                PaulsCustomer customer = new PaulsCustomer { Id = i, Name = "Customer " + i };
                list.Add(customer);
            }
            return list;
        }
    }

    DemoCustomerProvider customerProvider = new DemoCustomerProvider(numItems, fetchDelay);
    sample1.DataContext = new AsyncVirtualizingCollection<PaulsCustomer>(customerProvider, pageSize, pageTimeout);

In Vincent’s solution, you need to define a “Load” function somewhere in your ViewModel and pass that as a parameter to the collection. For example:

    private int Load(SortDescriptionCollection sortDescriptions, Predicate<object> filter, VincentsCustomer[] customers, int startIndex)
    {
        Thread.Sleep(fetchDelay);

        // Sorting
        bool isDescending = sortDescriptions != null && sortDescriptions.Count > 0 && sortDescriptions[0].Direction == ListSortDirection.Descending;
        int customerIndex;

        for (int i = 0; startIndex < numItems && i < customers.Length; ++i, ++startIndex)
        {
            customerIndex = isDescending ? numItems – startIndex – 1 : startIndex;
            customers[i] = new VincentsCustomer { Id = customerIndex, Name = "Customer " + customerIndex };
        }

        return numItems;
    }

    sample2.DataContext = new VirtualList<VincentsCustomer>(Load, 6 /* # of pages in memory at the same time */, pageSize);

Caching

Paul’s solution divides the original collection in pre-defined pages. When an item is accessed, Paul’s solution brings into memory the page that includes it, as well as the previous or next page, depending on whether the item belongs to the first or second half of its page. Paul also keeps track of the time each page is accessed. Every time a new item is retrieved, pages that haven’t been touched for a certain amount of time are discarded.

Vincent keeps a linked list of pages in memory. When a new page is brought into memory or an existing page is accessed, it is moved to the beginning of the list. If the number of pages exceeds a specified limit, the pages at the end of the list are discarded.

Sorting and filtering

Paul’s solution doesn’t address sorting and filtering. One way to add this functionality to his code is to change the FetchRange method of IItemsProvider to accept sorting and filtering information. In your implementation of IItemsProvider, you would implement FetchRange taking that information into account.

Because Vincent implements his own view, you can add SortDescriptions and a filter delegate as usual. Vincent takes care of passing this information to the Load method. It’s up to you to create a database query that incorporates that information in your implementation of Load.

Threading

Paul’s solution fetches the data in a second thread, therefore it doesn’t block the UI. The code that enables this scenario is in AsyncVirtualizingCollection<T>.

In Vincent’s solution, everything happens synchronously. This means that the UI will be unresponsive while the database is being queried.

DataTemplates

In Paul’s solution, we can use implicit DataTemplates as usual (a DataTemplate is “implicit” when it specifies a DataType but no key).

In Vincent’s solution, the items in the collection are of type DataRefBase<T>, which wraps your original type and provides property change notifications. Unfortunately, you can’t use an implicit template in this scenario because the type is generic. So, in this post’s project, I opted to use the DataTemplate explicitly when using Vincent’s solution (I refer to the DataTemplate using an explicit key).

(XAML 2009 adds support for generic types, but this feature is not yet available for compiled XAML scenarios. You can read more about this in Rob Relyea’s blog.)

Master-detail Scenario

Vincent’s solution works well when I implement a master-detail scenario by synchronizing the current and selected items:

Paul’s solution doesn’t work with the XAML above because WPF’s ListCollectionView calls IndexOf to track the current item, and Paul’s collection doesn’t support IndexOf. As an alternative, I implemented the master-detail scenario by binding the ContentControl’s Content to the ListView’s SelectedItem:

This second master-detail implementation works just as well as the first, and in fact, many people find the second solution easier to understand.

Selection

One shortcoming of Paul’s solution is the fact that a “collection reset” notification is raised every time a new page is loaded. As a result, selection is lost every time a new page is loaded. For example, if a user holds the down-arrow key to scroll quickly through a list, selection will jump back to the top of the list when a new page of data items is loaded.

This problem might be addressed by providing more granular collection change notifications when new data is available (look at the FireCollectionReset method in AsyncVirtualizingCollection<T>).

Summary

When comparing Paul’s solution to Vincent’s, you may prefer one over the other, depending which features you find valuable. An ideal solution would combine the best parts of both. Let me know if you come up with a good hybrid!

Data virtualization in Silverlight

Like WPF, Silverlight does not have built-in support for data virtualization based on scrolling. Unlike WPF, Silverlight has support for data virtualization through paging using PagedCollectionView. I won’t talk about PagedCollectionView here – instead I’ll focus the discussion on the scrolling based solution.

Vincent’s solution requires several features that are not present in the subset of .NET supported by Silverlight, but Paul’s solution compiles in Silverlight with very minor changes. However, the implementation of views in Silverlight is less optimized than in WPF, preventing this approach to data virtualization from working.

The internal implementation of views in WPF accesses only the items that it needs for display (as well as the count of the collection). This is what makes custom data virtualization solutions conceptually easy – we only need to keep in memory the data items that are accessed, and can discard all others.

The implementation of views in Silverlight, on the other hand, accesses all data items in the original collection at load time, independent of how many items it displays on the screen. This affects performance at load time in all scenarios, but because the delay is proportional to the count of the collection, it especially affects scenarios with large data sets. There is no way around this, other than re-implementing large portions of Silverlight’s collection views code. If you use .NET Reflector, you can look at this code in the InitializeSnapshot method of EnumerableCollectionView. This is, in my opinion, a big limitation of Silverlight that I am hoping will be addressed soon.

This fact increases the load time significantly, but by itself it shouldn’t cause the UI to hang for too long (depending on the number of items in the collection) because Paul’s solution loads pages asynchronously. However, because the collection is reset every time a new page is brought into memory, the effects of looping through all items are magnified. For example, if your collection has 100,000 items and the page size is 100 items, InitializeSnapshot’s loop will bring 1,000 pages into memory at load time, asynchronously. As each of those pages is ready to be loaded, a collection reset is triggered, causing Silverlight to loop through all 100,000 items again, causing all pages to be loaded into memory again. At the end of this post, I provide a link to a Silverlight project with Paul’s data virtualization implementation so you can see it hang and debug it in case you’re curious.

I am not aware of any custom data virtualization solution based on scrolling that works with Silverlight at this point, so here’s a fun challenge for you! Please send me email if you come up with a solution for this problem.

You can run an xbap showing both data virtualization solutions side by side, and you can download the corresponding WPF project.

You can also download a Silverlight project that shows data virtualization *not* working.

Update: If you’re looking for a data virtualization solution for WPF, make sure you also read my more recent post.

Posted by Bea under Silverlight, WPF | Comments (28)

Next Page »