Skip to content

Conversation

@ideas-into-software
Copy link

@ideas-into-software ideas-into-software commented Feb 5, 2024

Problems

HL7 FIHR model related

  1. Type-specific XSD schemas (e.g. 'patient.xsd') are not self-contained, reference all other types, and all reside under one namespace – it makes no difference whether EMF model is generated from such type-specific XSD schema ('patient.xsd') or main schema containing all types;

  2. EMF Ecore generated off such XSD schema contains close to 1400 eClassifier definitions (EClasses, Enums, EDataTypes);

  3. Java code generated off such EMF Ecore reaches beyond limits of Java language – exceeding the 65535 bytes limit for initializers and methods as well as 65536 entries constant pool limit, i.e.:

  • “The code for the static initializer is exceeding the 65535 bytes limit” (see org.hl7.fhir.JurisdictionValueSetEnum in org.avatar.fhir.model.core bundle);

  • “The code of method doSwitch(int, EObject) is exceeding the 65535 bytes limit” (see org.hl7.fhir.util.FhirSwitch.doSwitch(int, EObject) in org.avatar.fhir.model.core bundle);

  • “The code of method createPackageContents() is exceeding the 65535 bytes limit” (see org.hl7.fhir.impl.FhirPackageImpl.createPackageContents() in org.avatar.fhir.model.core bundle);

  • “The code of method initializePackageContents() is exceeding the 65535 bytes limit” (see org.hl7.fhir.impl.FhirPackageImpl.initializePackageContents() in org.avatar.fhir.model.core bundle);

  • “Too many constants, the constant pool for FhirPackageImpl would exceed 65536 entries” (see org.hl7.fhir.impl.FhirPackageImpl in org.avatar.fhir.model.core bundle);

Screenshot from 2024-02-06 00-02-36

  1. ‘Patient’ EClass does not extend EObject directly – is part of 4-level deep class hierarchy;

  2. ‘Patient’ EClass does not define any attribute which can be used as ID – only in one of the types it inherits from such is declared, but not as attribute but as reference to specialized EClass called 'ID'; this results in "The given EObject of EClass Patient doesn't contain a id attribute but should have one" error when attempting to save such to Mongo database;

EMF related

  • FIHR EMF Generator Model with “Load initialization” option turned on (loadInitialization=”true”) results in code which does not work – results in “java.lang.RuntimeException: Missing serialized package: xhtml.ecore” error; this is perhaps related to wrong path to ECORE package defined in packageFilename of *PackageImpl (i.e. without /model/ prefix); if “Load Initialization” did work, there would be fewer errors related to Java language limits and all these could be handled on EMF code generator side by splitting large methods into sub-routines / initializing static fields via static methods;

Screenshot from 2024-01-31 00-30-22
Screenshot from 2024-01-31 01-50-15

java.lang.ExceptionInInitializerError
	at org.hl7.fhir.impl.FhirPackageImpl.init(FhirPackageImpl.java:9773)
	at org.hl7.fhir.FhirPackage.<clinit>(FhirPackage.java:68)
	at org.hl7.fhir.configuration.FhirConfigurationComponent.activate(FhirConfigurationComponent.java:58)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.apache.felix.scr.impl.inject.methods.BaseMethod.invokeMethod(BaseMethod.java:245)
	at org.apache.felix.scr.impl.inject.methods.BaseMethod.access$500(BaseMethod.java:41)
	at org.apache.felix.scr.impl.inject.methods.BaseMethod$Resolved.invoke(BaseMethod.java:687)
	at org.apache.felix.scr.impl.inject.methods.BaseMethod.invoke(BaseMethod.java:531)
	at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:317)
	at org.apache.felix.scr.impl.inject.methods.ActivateMethod.invoke(ActivateMethod.java:307)
	at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:354)
	at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:115)
	at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:1002)
	at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:975)
	at org.apache.felix.scr.impl.manager.AbstractComponentManager.activateInternal(AbstractComponentManager.java:785)
	at org.apache.felix.scr.impl.manager.AbstractComponentManager.enableInternal(AbstractComponentManager.java:674)
	at org.apache.felix.scr.impl.manager.AbstractComponentManager.enable(AbstractComponentManager.java:437)
	at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.enableComponents(ConfigurableComponentHolder.java:671)
	at org.apache.felix.scr.impl.BundleComponentActivator.initialEnable(BundleComponentActivator.java:310)
	at org.apache.felix.scr.impl.Activator.loadComponents(Activator.java:593)
	at org.apache.felix.scr.impl.Activator.access$200(Activator.java:74)
	at org.apache.felix.scr.impl.Activator$ScrExtension.start(Activator.java:460)
	at org.apache.felix.scr.impl.AbstractExtender.createExtension(AbstractExtender.java:196)
	at org.apache.felix.scr.impl.AbstractExtender.modifiedBundle(AbstractExtender.java:169)
	at org.apache.felix.scr.impl.AbstractExtender.modifiedBundle(AbstractExtender.java:49)
	at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:488)
	at org.osgi.util.tracker.BundleTracker$Tracked.customizerModified(BundleTracker.java:420)
	at org.osgi.util.tracker.AbstractTracked.track(AbstractTracked.java:232)
	at org.osgi.util.tracker.BundleTracker$Tracked.bundleChanged(BundleTracker.java:450)
	at org.apache.felix.framework.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:915)
	at org.apache.felix.framework.EventDispatcher.fireEventImmediately(EventDispatcher.java:834)
	at org.apache.felix.framework.EventDispatcher.fireBundleEvent(EventDispatcher.java:516)
	at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4847)
	at org.apache.felix.framework.Felix.startBundle(Felix.java:2363)
	at org.apache.felix.framework.BundleImpl.start(BundleImpl.java:1006)
	at aQute.launcher.Launcher.start(Launcher.java:699)
	at aQute.launcher.Launcher.startBundles(Launcher.java:679)
	at aQute.launcher.Launcher.activate(Launcher.java:585)
	at aQute.launcher.Launcher.launch(Launcher.java:403)
	at aQute.launcher.Launcher.run(Launcher.java:185)
	at aQute.launcher.Launcher.main(Launcher.java:161)
	at aQute.launcher.pre.EmbeddedLauncher.executeWithRunPath(EmbeddedLauncher.java:169)
	at aQute.launcher.pre.EmbeddedLauncher.findAndExecute(EmbeddedLauncher.java:134)
	at aQute.launcher.pre.EmbeddedLauncher.main(EmbeddedLauncher.java:51)
Caused by: java.lang.RuntimeException: Missing serialized package: xhtml.ecore
	at org.w3._1999.xhtml.impl.XhtmlPackageImpl.loadPackage(XhtmlPackageImpl.java:8930)
	at org.w3._1999.xhtml.impl.XhtmlPackageImpl.init(XhtmlPackageImpl.java:791)
	at org.w3._1999.xhtml.XhtmlPackage.<clinit>(XhtmlPackage.java:437)
	... 47 more
  • XMLNamespace EMF Generator Model, referenced from XHTML schema, cannot be read from org.eclipse.emf.ecore JAR file, which required duplicating it in workspace as org.avatar.fhir.model.namespace; however, this in return resulted in a different error related to namespace clash and required temporarily redefining its nsURI to "http://www.w3.org/XML/2024/namespace", until XMLNamespace EMF genmodel contained in org.eclipse.emf.ecore JAR can be referenced directly;

Screenshot from 2024-01-26 19-51-54

HAPI FHIR (Java API for HL7 FHIR Clients and Servers) related

"HAPI FHIR – Java API for HL7 FHIR Clients and Servers" cannot be used with tech stack required to use for AVATAR Libraries project:

  • Usage of Javax, not Jakarta;
  • EMF model would need to use (implement or extend) specialized interfaces - e.g. org.hl7.fhir.instance.model.api.IbaseResource;
  • EMF model would need to use specialized annotations;

Gecko related

REST API methods do not return responses in XML format

Currently, all REST API methods defined on org.avatar.fhir.rest.PatientResource return responses in JSON format, despite being annotated as producing XML media type, and specifying application/xml request header;

EMFRepository does not support storing EObjects of EClasses which do not define ID attribute directly

  • E.g. 'Patient' EClass (org.hl7.fhir_patient.Patient) does not define any attribute which can be used as ID – only in one of the types in inherits from such is declared, but not as attribute but as reference to specialized EClass called 'ID';
  • Such combination resulted in "The given EObject of EClass Patient doesn't contain a id attribute but should have one" error;
  • It appears AbstractEMFMongoRepository expects ID attribute directly on EClass of EObject being saved – it does not take into consideration attributes of references defined on types inherited from, i.e. value attribute of org.hl7.fhir_patient.Id reference defined on org.hl7.fhir.Resource, from which org.hl7.fhir.Patient inherits;
java.lang.IllegalStateException: The given EObject of EClass Patient doesn't contain a id attribute but should have one
	at org.gecko.emf.repository.mongo.impl.AbstractEMFMongoRepository.createUri(AbstractEMFMongoRepository.java:168)
	at org.gecko.emf.repository.DefaultEMFRepository.createResource(DefaultEMFRepository.java:489)
	at org.gecko.emf.repository.DefaultEMFRepository.attach(DefaultEMFRepository.java:232)
	at org.gecko.emf.repository.DefaultEMFRepository.save(DefaultEMFRepository.java:175)
	at org.gecko.emf.repository.DefaultEMFRepository.save(DefaultEMFRepository.java:253)
	at org.gecko.emf.repository.DefaultEMFRepository.save(DefaultEMFRepository.java:262)
	at org.avatar.fhir.service.patient.impl.PatientServiceImpl.savePatient(PatientServiceImpl.java:62)

EPushStreamProvider does not support creating push streams off EClasses which are part of a complex class hierarchy

Currently, it appears EPushStreamProvider does not support creating push streams off EClasses which are part of a complex class hierarchy and do not extend EObject directly, as ClassCastException is thrown during index initialization when using EPushStreamProvider with such EClass (org.hl7.fhir_patient.Patient);

 java.lang.ClassCastException: class org.hl7.fhir_patient.impl.PatientImpl cannot be cast to class org.gecko.emf.pushstream.EPushStreamProvider (org.hl7.fhir_patient.impl.PatientImpl is in unnamed module of loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @72fd8a3c; org.gecko.emf.pushstream.EPushStreamProvider is in unnamed module of loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @fb49fdf)

 at org.avatar.fhir.service.patient.impl.PatientIndexServiceImpl.initializeIndex(PatientIndexServiceImpl.java:109)

 at org.avatar.fhir.service.patient.impl.PatientIndexServiceImpl.lambda$0(PatientIndexServiceImpl.java:85)

 at org.osgi.util.promise.DeferredPromiseImpl$Submit.run(DeferredPromiseImpl.java:767)

 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)

 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)

 at java.base/java.lang.Thread.run(Thread.java:833)

EMF validation of payloads passed to REST API methods does not work in JakartaRS

Currently, for all REST API methods containing message body (POST and PUT), defined on org.avatar.fhir.rest.PatientResource, content validation does not work in JakartaRS – same exact configuration worked with JaxRS;

Solutions

HL7 FIHR model related

Modify FHIR Ecore generated off XSD schema in the following way:

  • from 'DomainResource' EClass drop 'contained' reference to ResourceContainer – it is through this reference that most of the other 1400 eClassifiers (EClasses, Enums, EDataTypes) are pulled in to the Patient;
  • from 'Bundle' Eclass: drop 'issues' reference to 'ResourceContainer';
  • from 'BundleEntry' Eclass: drop 'resource' reference to 'ResourceContainer';
  • from 'BundleResponse' Eclass: drop 'outcome' reference to 'ResourceContainer';
  • from 'ParametersParameter' Eclass: drop 'resource' reference to 'ResourceContainer';
  • drop all “orphaned” Eclassifers, i.e. 1194 in total;

This exact solution was applied in org.avatar.fhir.model.patient bundle, while full FIHR Ecore along with problems described is present in org.avatar.fhir.model.core bundle.

EMF related

  • Modify FHIR Ecore generated off XSD as outlined in previous section;
  • Temporarily create org.avatar.fhir.model.namespace bundle holding XMLNamespace model, and modify its nsURI to "http://www.w3.org/XML/2024/namespace", until XMLNamespace EMF genmodel contained in org.eclipse.emf.ecore JAR can be referenced directly;

Gecko related

  • Temporarily define fakeId attribute directly on org.hl7.fhir.Patient EClass and mark it as ID – apparently, this alters the model itself and duplicates information already present, and should only be used temporarily until underlying issue of EMFRepository lack of support for storing EObjects of EClasses which do not define ID attribute directly is addressed;

Screenshot from 2024-02-03 02-50-59

  • Temporarily skip using EPushStreamProvider and query Mongo database directly to initialize index – this solution is present in org.avatar.fhir.service.patient.impl.PatientIndexServiceImpl.initializeIndex() method, while the EPushStreamProvider problem can be observed in org.avatar.fhir.service.patient.impl.PatientIndexServiceImpl.initializeIndex_EPushStreamProvider() method;

  • Temporarily disable EMF validation for all REST API methods containing message body (POST and PUT), defined on org.avatar.fhir.rest.PatientResource;

Questions

I. What does 'patient file' refer to?

II.

  1. Which version of HL7 FHIR specification should be supported – only latest one (R5) or previous versions as well and if so, which?

  2. Can it be expected that servers from which health providers will connect to this system will support latest version (R5)?

III. Should "Capability Statement" be implemented?

Based on https://www.hl7.org/fhir/capabilitystatement.html and https://www.hl7.org/fhir/profiling.html#api, apparently it is required in order for such solution to be compliant with HL7 FHIR standard;

IV. Should security (authentication / authorization) be implemented? eg. via Keycloak?

Based on https://www.hl7.org/fhir/security.html#6.1.0, it seems it is required in order for such solution to be compliant with HL7 FHIR standard;

V. Should REST API endpoint methods defined on org.avatar.fhir.rest.PatientResource be fully compliant with HL7 FHIR standard or such basic implementation is enough?

Based on https://www.hl7.org/fhir/http.html, it seems exact implementation of these CRUD methods as per this spec is required for such solution to be compliant with HL7 FHIR standard;

VI. Should record versioning be supported?

Based on https://www.hl7.org/fhir/overview-dev.html and https://www.hl7.org/fhir/resource.html, record version is one of the required elements of resource (including 'Patient' resource) and is used for tracking changes to resources, and preventing changes overwriting each other;

VII. Beyond annotating REST API methods as producing XML media type, and specifying application/xml request header, is there some additional requirement for responses returned to use XML format?

Currently, all REST API methods defined on org.avatar.fhir.rest.PatientResource return responses in JSON format, ignoring application/xml request header;

VIII. Does Gecko EMF Mongo repository (org.gecko.emf.repository.EMFRepository) support storing EObjects of EClasses which do not define ID attribute directly?

  • E.g. 'Patient' EClass (org.hl7.fhir_patient.Patient) does not define any attribute which can be used as ID – only in one of the types it inherits from such is declared, but not as attribute but as reference to specialized EClass called 'ID';
  • Such combination resulted in "The given EObject of EClass Patient doesn't contain a id attribute but should have one" error;
  • It appears AbstractEMFMongoRepository expects ID attribute directly on EClass of EObject being saved – it does not take into consideration attributes of references defined on types inherited from, i.e. value attribute of org.hl7.fhir_patient.Id reference defined on org.hl7.fhir.Resource, from which org.hl7.fhir.Patient inherits;
  • Temporary solution was to define fakeId attribute directly on org.hl7.fhir.Patient EClass and mark it as ID, but this alters the model itself and duplicates information already present.
java.lang.IllegalStateException: The given EObject of EClass Patient doesn't contain a id attribute but should have one
	at org.gecko.emf.repository.mongo.impl.AbstractEMFMongoRepository.createUri(AbstractEMFMongoRepository.java:168)
	at org.gecko.emf.repository.DefaultEMFRepository.createResource(DefaultEMFRepository.java:489)
	at org.gecko.emf.repository.DefaultEMFRepository.attach(DefaultEMFRepository.java:232)
	at org.gecko.emf.repository.DefaultEMFRepository.save(DefaultEMFRepository.java:175)
	at org.gecko.emf.repository.DefaultEMFRepository.save(DefaultEMFRepository.java:253)
	at org.gecko.emf.repository.DefaultEMFRepository.save(DefaultEMFRepository.java:262)
	at org.avatar.fhir.service.patient.impl.PatientServiceImpl.savePatient(PatientServiceImpl.java:62)
       (...)

IX. Does Gecko EMF Push Stream Provider (EPushStreamProvider) support creating push streams off EClasses which are part of a complex class hierarchy and do not extend EObject directly?

Currently, it appears it is not supported, as ClassCastException is thrown during index initialization when using EPushStreamProvider with EClass which is part of a complex class hierarchy and does not extend EObject directly (org.hl7.fhir_patient.Patient)

java.lang.ClassCastException: class org.hl7.fhir_patient.impl.PatientImpl cannot be cast to class org.gecko.emf.pushstream.EPushStreamProvider (org.hl7.fhir_patient.impl.PatientImpl is in unnamed module of loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @72fd8a3c; org.gecko.emf.pushstream.EPushStreamProvider is in unnamed module of loader org.apache.felix.framework.BundleWiringImpl$BundleClassLoader @fb49fdf)
	at org.avatar.fhir.service.patient.impl.PatientIndexServiceImpl.initializeIndex(PatientIndexServiceImpl.java:109)
	at org.avatar.fhir.service.patient.impl.PatientIndexServiceImpl.lambda$0(PatientIndexServiceImpl.java:85)
	at org.osgi.util.promise.DeferredPromiseImpl$Submit.run(DeferredPromiseImpl.java:767)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

X. Does EMF validation of payloads passed to REST API methods work in JakartaRS?

Currently, for all REST API methods containing message body (POST and PUT), defined on org.avatar.fhir.rest.PatientResource, content validation does not work in JakartaRS – same exact configuration worked with JaxRS;

XI. Should EMF-repository based version of #toEObject method be included in org.gecko.search.util.DocumentUtil, instead of being duplicated across projects?

  • Current version of org.gecko.search.util.DocumentUtil.toEObject method expects ENTIRE EMF EObject to be stored in the Lucene index, which is completely unrealistic – in any type of commercial / production application such approach quickly results in Lucene index becoming unusable, both for searching and opening;
  • Proper approach should be to only index fields which are searched on, along with ECLASS and ID value of such EObject, and once it’s found in the index, EMF repository should be queried to load it;
  • Ever since that problem was discovered in the PlayerTour application backend, custom #toEObject method implementation is using such approach. However, such is NOT YET available in org.gecko.search.util.DocumentUtil, therefore it must be duplicated, for example in: org.avatar.fhir.service.patient.impl.PatientSearchServiceImpl.toEObject(Document, EMFRepository);

XII. Can org.gecko.search.util.DocumentUtil.ECLASS_URI be made public, instead of being duplicated across projects when ECLASS_URI has to be stored during index creation?

E.g. duplicated again as org.avatar.fhir.service.patient.impl.PatientIndexHelper.FIELD__ECLASS_URI, previously in PlayerTour project, and each time approach of storing in Lucene index only fields searched on is used (see question regarding #toEObject method for more details);

XIII. Considering Java language limits triggered by full HL7 FIHR model when using static Ecore approach, should dynamic Ecore approach be used instead?

XIV. Considering all these issues and time needed to fix those, should a fully compliant backend using "HAPI FHIR – Java API for HL7 FHIR Clients and Servers" (https://github.com/hapifhir/hapi-fhir) be set up then?

Signed-off-by: Michael H. Siemaszko <mhs@into.software>
Signed-off-by: Michael H. Siemaszko <mhs@into.software>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant