Unfortunately, I still have to deal with flat files quite a bit when dealing with business partners. Flat files aren’t all bad.. they’re easily understood and somewhat effecient to transmit payloads between disperate systems (when compared to XML at least). In the past, I’ve rolled out my own flat file mapping systems but I’ve recently caught a glimpse of Spring Batch and I must say that I like what I see.
For me, Mule and Spring Batch seems like a natural fit for processing data that must be recieved or delivered as a flat file. I created a couple of Mule transfomers based upon the framework:
| FlatFileToBeans | converts your flat file to a list of java beans |
| BeansToFlatFile | converts a list of java beans to a flat file |
FlatFileToBeans.groovy
public class FlatFileToBeans extends AbstractTransformer {
private static final def log = LogFactory.getLog(FlatFileToBeans)
ObjectPool pool
FlatFileToBeans() {
super.registerSourceType(byte[].class)
super.registerSourceType(InputStream.class)
}
protected Object doTransform(Object src, String encoding) {
FlatFileItemReader reader = null
try {
reader = (FlatFileItemReader) pool.borrowObject()
reader.encoding = encoding
reader.resource = convertToresource(src)
return transform(reader)
}
finally {
pool.returnObject(reader)
}
}
protected Resource convertToresource(Object src) {
return src instanceof InputStream ? new InputStreamResource(src) : new ByteArrayResource(src)
}
protected def transform(FlatFileItemReader reader) {
reader.open(new ExecutionContext())
def beans = []
def bean
while (bean = reader.read()) {
beans += bean
}
return beans
}
}
BeansToFlatFile.groovy
public class BeansToFlatFile extends AbstractTransformer {
private static final def log = LogFactory.getLog(BeansToFlatFile)
ObjectPool pool
String headers
String lineDelimeter = "\r\n"
BeansToFlatFile() {
super.registerSourceType(List)
}
byte[] transform(List domainObjects) {
LineAggregator aggregator = null
try {
aggregator = (LineAggregator)pool.borrowObject()
StringBuffer sb = new StringBuffer()
if (headers) {
sb.append("${headers}${lineDelimeter}")
}
domainObjects.each {bean ->
String line = aggregator.aggregate(bean)
sb.append("${line}${lineDelimeter}")
}
String response = sb.toString()
if (log.isTraceEnabled()) {
log.trace("Flat file output data:\n ${response}")
}
return response.bytes
} finally {
pool.returnObject(aggregator)
}
}
protected Object doTransform(Object src, String encoding) {
return transform(src)
}
}
The real magic here is in the wiring (of course). You might also notice that I’m using a commons object pool. Mule transformers are not thread safe (learned this the hard way). Each transformer is created only once per endpoint and is shared for each thread. Orginally, I synchronized my transformer but that’s not really performant. Instead, I created a quick object factory that make suse of the spring bean factory to create objects for the pool:
SpringBeanPoolableObjectFactory.groovy
public class SpringBeanPoolableObjectFactory extends BasePoolableObjectFactory implements BeanFactoryAware {
String beanName
BeanFactory beanFactory
public Object makeObject() {
if (beanFactory.isSingleton(beanName)) {
throw new InvalidObjectPoolException("${beanName} is configured as a singleton and cannot be pooled")
}
return beanFactory.getBean(beanName)
}
}
Now, you can wire it all together. Here’s an example of both:
springbatch-config.xml
<bean id="fooheaders" class="java.lang.String"> <constructor-arg value="name, description, comments"/> </bean> <bean class="com.acme.model.Foo" id="foo" scope="prototype"/> <bean id="fooFileItemReader" scope="prototype" class="org.springframework.batch.item.file.FlatFileItemReader"> <property name="lineMapper"> <bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper"> <property name="lineTokenizer"> <bean class="org.springframework.batch.item.file.transform.FixedLengthTokenizer"> <property name="names" value="name, description, comments"/> <property name="columns" value="1-4,5-12,13-22"/> </bean> </property> <property name="fieldSetMapper"> <bean class="org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper"> <property name="prototypeBeanName" value="foo"/> </bean> </property> </bean> </property> </bean> <bean id="baseBeanPool" abstract="true" class="org.apache.commons.pool.impl.GenericObjectPool"> <property name="whenExhaustedAction"> <util:constant static-field="org.apache.commons.pool.impl.GenericObjectPool.WHEN_EXHAUSTED_BLOCK"/> </property> <property name="maxWait" value="30000"/> <property name="maxActive" value="20"/> </bean> <bean id="fooFileItemReaderPool" class="org.apache.commons.pool.impl.GenericObjectPool" parent="baseBeanPool"> <property name="factory"> <bean class="com.sonicdrivein.esb.share.pool.SpringBeanPoolableObjectFactory"> <property name="beanName" value="fooFileItemReader"/> </bean> </property> </bean> <bean id="fooLineAggregator" scope="prototype" class="org.springframework.batch.item.file.transform.DelimitedLineAggregator"> <property name="delimiter" value=","/> <property name="fieldExtractor"> <bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor"> <property name="names" ref="fooheaders"/> </bean> </property> </bean> <bean id="fooLineAggregatorPool" class="org.apache.commons.pool.impl.GenericObjectPool" parent="baseBeanPool"> <property name="factory"> <bean class="com.sonicdrivein.esb.share.pool.SpringBeanPoolableObjectFactory"> <property name="beanName" value="fooLineAggregator"/> </bean> </property> </bean>
mule-config.xml
<spring:beans>
<spring:import resource="classpath:springbatch-config.xml"/>
</spring:beans>
<custom-transformer name="flatFileToFoo" class="com.sonicdrivein.esb.share.transformer.FlatFileToBeans">
<spring:property name="pool" ref="fooFileItemReaderPool"/>
</custom-transformer>
<custom-transformer name="fooToFlatFile" class="com.sonicdrivein.esb.share.transformer.BeansToFlatFile">
<spring:property name="pool" ref="fooLineAggregatorPool"/>
<spring:property name="headers" ref="fooheaders"/>
</custom-transformer>
I see myself getting a lot of reuse out of these transformers and I hope you do too 
