Facing new challenges everyday

Using a FSIS CTS flow to generate text files and compile as ESP dictionaries

December 23rd, 2011

After have some trouble trying while create my FSIS CTS flow, I could finally read the data need from SQL Server instance and push to ESP for indexing, once I got everything working as expected I started to think in what else I could easily to in a CTS flow.

My next challenge was create a ESP dictionary for use on query completion server of ESP, the majority of built in operators doesn’t provide a easy way to do it, hopefully I found the RunCode operator that I can use to perform anything that I need with a custom C# or VB code.

Once I found enough examples about how a RunCode works, I created my own version that take all records read by DatabaseReader operator and then write all string collumn in each line of text file to be compiled with dictcompile utility.

My current CTS flow that read data from SQL Server to feed ESP and write text file to compile a ESP dictionary is something like this:

You can find the operator code below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92

// To customize the RunCode operator, edit the four customizable sections:
// 1. The import list
// 2. The Initialize method
// 3. The Execute method
// 4. The Finalize method

namespace GeneratedCode {
using System;
using System.IO;
using System.Collections;
using System.Collections.Generic;
using System.Data;
using System.Globalization;
using System.Net;
using System.Text;
using System.Xml;
using System.Xml.Linq;

// [RunCode]
internal sealed class RunCode {

private RunCode() {
}

#region Initialize
/// <summary>
/// Performs initialization logic for the custom operation.
/// </summary>
/// <param name="context">
/// Represents a data context that is shared between the Initialize, Execute and Finalize methods.
/// </param>
private static void Initialize(Microsoft.Ceres.ContentEngine.Processing.BuiltIn.RunCode.RunCodeContext context) {
// [BeginInitialize]
string fileName = @"c:\temp\locations.txt";
StreamWriter sw = File.CreateText(fileName);
context["writer"] = sw;
context["terms"] = new ArrayList();
// [EndInitialize]
}
#endregion

#region Execute
/// <summary>
/// Executes the main logics of this RunCode operator.
/// </summary>
/// <param name="input">
/// Contains the input from the preceding operator.
/// </param>
/// <param name="output">
/// When the method returns, contains the output from the current operator; this will serve as input for the following operator.
/// </param>
/// <param name="context">
/// Represents a data context that is shared between the Initialize, Execute and Finalize methods.
/// </param>
private static void Execute(object[] input, object[] output, Microsoft.Ceres.ContentEngine.Processing.BuiltIn.RunCode.RunCodeContext context) {
string city = ((string)(input[0]));
string state = ((string)(input[1]));
string contry = ((string)(input[2]));
// [BeginMain]
ArrayList terms = (ArrayList) context["terms"];
StreamWriter sw = (StreamWriter) context["writer"];
foreach(String item in input) {
if(! terms.Contains(item)) {
sw.WriteLine(item);
terms.Add(item);
}
}
// [EndMain]
output[0] = city;
output[1] = state;
output[2] = contry;
}
#endregion

#region Finalize
/// <summary>
/// Performs finalization logic for the custom operation.
/// </summary>
/// <param name="context">
/// Represents a data context that is shared between the Initialize, Execute and Finalize methods.
/// </param>
private static void Finalize(Microsoft.Ceres.ContentEngine.Processing.BuiltIn.RunCode.RunCodeContext context) {
// [BeginFinalize]
StreamWriter sw = (StreamWriter) context["writer"];
sw.Close();
sw.Dispose();
// [EndFinalize]
}
#endregion
}
}

Once you have suggestions.txt generated, you will be able to compile it as ESP dictionary with the following command:

dictcompile -i suggestions.txt -t ED -o suggestions.aut