Logo Oracle Deutschland   Deutschsprachige APEX und PL/SQL Community

REST Services and Application Express 5.1: Filtering

Released APEX version Database version
Juli 2017 ab 5.1 ab 11.2

More and more Application Express developers are faced with the requirement to integrate REST services or HTTP/JSON data feeds into their applications. Application Express provides great support for SOAP web services, but for REST services using JSON to exchange data, the built-in functionality is limited. For instance, all the JSON parsing has to be done manually by employing PL/SQL code or SQL functions.

During the last weeks, we released a few articles describing how to access a REST service from within Application Express, how to leverage the REST Client Assistant packaged application and how to deal with REST services providing large result sets ...

Packaged Application REST Client Assistant

Packaged Application REST Client Assistant


Packaged Application REST Client Assistant

Packaged Application REST Client Assistant

In the previous articles, we always fetched all response data from the REST service - either all at once or page-wise, depending on REST service capabilities. But in practice, it's often required to get only a subset of the response data - we want to filter the response based on some end user criteria.

"So that's easy" might be your first thought - since all JSON parsing is done with SQL functions. In the SQL query, both XMLTABLE and JSON_TABLE act as a row source, so its result can be treated like rows from a table or view. Thus we can simply amend the WHERE clause of the SQL query, which has been generated be the REST Client Assistant packaged application. Let's illustrate that with the "Earthquake" example from the first article Generate SQL and PL/SQL code to access REST services using REST Client Assistant. In that article, REST service data was displayed as an Application Express Classic Report with the following SQL query.

select 
    j."TYPE", 
    j."MAG", 
    j."PLACE", 
    :
    j."ID"
from apex_collections c, json_table(
    c.clob001 format json,
    '$.features[*]'
    columns (
        "TYPE"      VARCHAR2(4000)  path '$.type',
        "MAG"       NUMBER          path '$.properties.mag',
        "PLACE"     VARCHAR2(4000)  path '$.properties.place',
        :
        "ID"        VARCHAR2(4000)  path '$.id'
    )
) j
where c.collection_name = 'REST_COLLECTION'

Now it's more than easy to add some filtering to the WHERE clause, as follows:

select 
    j."TYPE", 
    j."MAG", 
    j."PLACE", 
    :
    j."ID"
from apex_collections c, json_table(
    c.clob001 format json,
    '$.features[*]'
    columns (
        "TYPE"      VARCHAR2(4000)  path '$.type',
        "MAG"       NUMBER          path '$.properties.mag',
        "PLACE"     VARCHAR2(4000)  path '$.properties.place',
        :
        "ID"        VARCHAR2(4000)  path '$.id'
    )
) j
where c.collection_name = 'REST_COLLECTION'
  and j.mag             > 2 -- hier sind natürlich auch APEX-Items denkbar, bspw. :P1_MAG

And, of course, we can add Application Express Page Items here. So, it's pretty straightforward to add page items to the Application Express page and to add these to the SQL WHERE clause. End user filtering for a REST Service - done. So far, so good.

This was so easy, because the Earthquake example does not return huge amount of rows and because it does not return its data page-wise. Imagine a REST service returning a huge amount of data (thousands or even millions of rows). As learned in the Work with REST services returning large result sets page-wise article, these services return their data page-wise. So, in order to apply a SQL filter, we would first have to fetch all data, walking though all the response pages. Then all data would be buffered in the database and we would be able to apply the SQL filter. Of course, we could stop when we have found a sufficient amount of rows, but perhaps we'll find our rows of interest at the very last page of the REST service response.

To get this straight: That SQL filtering approach will only work for REST services returning a reasonable amount of rows and all at once. For larger response sizes, that would lead to unacceptable performance and response times.

The goal must be, to execute a filter on the REST service side. For REST services provided by Oracle REST Data Services (ORDS), a filter syntax is provided: Filter criteria can be passed to ORDS with the REST request and will be executed server-side returning only the final (and limited) amount of rows. The following paragraphs contain a few examples to illustrate how this filter syntax can be used.

But this ORDS filter syntax is not a common standard - it's an ORDS feature. Whether a REST service even provides filtering capabilities, and if yes, how the syntax looks like, varies from REST service to REST service. So when having a requirement to filter data from a REST service, the first exercise should be to examine whether the REST service supports server-side filtering and its syntax. The rest of the articale thus assumes that we have REST services powered by Oracle REST Data Services.

Detailed information about the ORDS filter syntax is available in the ORDS documentation. Filters are passed using a JSON syntax either as the "q" query string parameter or as the request body for a POST request.

As an example, we'll now work with a REST service returning the well-known EMP table data, as described in the Work with REST services returning large result sets page-wise article. It may be available on http://{server}:{port}/ords/scott/emp/. Following, you'll find three SQL queries doing it all at once: APEX_WEB_SERVICE invokes the REST service (passing the filter), and JSON_TABLE parses the response.

1. Get all data (no filter)

select 
    j."EMPNO", 
    j."ENAME", 
    j."JOB", 
    j."MGR", 
    to_date( j."HIREDATE", 'YYYY-MM-DD"T"HH  :MI:SS"Z"' ) as "HIREDATE", 
    j."SAL", 
    j."COMM", 
    j."DEPTNO"
from json_table(
    apex_web_service.make_rest_request(
        'http://{server}:{port}/ords/scott/emp/',
        'GET' ),
    '$.items[*]'
    columns (
        "EMPNO"      NUMBER          path '$.empno',
        "ENAME"      VARCHAR2(4000)  path '$.ename',
        "JOB"        VARCHAR2(4000)  path '$.job',
        "MGR"        NUMBER          path '$.mgr',
        "HIREDATE"   VARCHAR2(4000)  path '$.hiredate',
        "SAL"        NUMBER          path '$.sal',
        "COMM"       NUMBER          path '$.comm',
        "DEPTNO"     NUMBER          path '$.deptno'
    )
) j
/

EMPNO  ENAME   JOB        MGR   HIREDATE             SAL   COMM  DEPTNO  
------ ------- ---------- ----- -------------------- ----- ----- ------
7369   SMITH   CLERK      7902  17.12.1980 08:00:00  800         20      
7499   ALLEN   SALESMAN   7698  20.02.1981 08:00:00  1600  300   30      
7521   WARD    SALESMAN   7698  22.02.1981 08:00:00  1250  500   30      
7566   JONES   MANAGER    7839  02.04.1981 08:00:00  2975        20      
7654   MARTIN  SALESMAN   7698  28.09.1981 07:00:00  1250  1400  30      
7698   BLAKE   MANAGER    7839  01.05.1981 07:00:00  2850        30      
7782   CLARK   MANAGER    7839  09.06.1981 07:00:00  2450        10      
7788   SCOTT   ANALYST    7566  09.12.1982 08:00:00  3000        20      
7839   KING    PRESIDENT        17.11.1981 08:00:00  5000        10      
7844   TURNER  SALESMAN   7698  08.09.1981 07:00:00  1500  0     30      
7876   ADAMS   CLERK      7788  12.01.1983 08:00:00  1100        20      
7900   JAMES   CLERK      7698  03.12.1981 08:00:00  950         30      
7902   FORD    ANALYST    7566  03.12.1981 08:00:00  3000        20      
7934   MILLER  CLERK      7782  23.01.1982 08:00:00  1300        10    

14 rows selected.

2. Filter for DEPTNO Only DEPTNO 30 rows are of interest.

select 
    j."EMPNO", 
    j."ENAME", 
    j."JOB", 
    j."MGR", 
    to_date( j."HIREDATE", 'YYYY-MM-DD"T"HH  :MI:SS"Z"' ) as "HIREDATE", 
    j."SAL", 
    j."COMM", 
    j."DEPTNO"
from json_table(
    apex_web_service.make_rest_request(
        'http://{server}:{port}/ords/scott/emp/?q=' ||
            utl_url.escape( '{"deptno": { "$eq": 30 } }' ),
        'GET' ),
    '$.items[*]'
    columns (
        "EMPNO"      NUMBER          path '$.empno',
        "ENAME"      VARCHAR2(4000)  path '$.ename',
        "JOB"        VARCHAR2(4000)  path '$.job',
        "MGR"        NUMBER          path '$.mgr',
        "HIREDATE"   VARCHAR2(4000)  path '$.hiredate',
        "SAL"        NUMBER          path '$.sal',
        "COMM"       NUMBER          path '$.comm',
        "DEPTNO"     NUMBER          path '$.deptno'
    )
) j
/

EMPNO  ENAME   JOB       MGR   HIREDATE             SAL   COMM  DEPTNO  
------ ------- --------- ----- -------------------- ----- ----- ------
7499   ALLEN   SALESMAN  7698  20.02.1981 08:00:00  1600  300   30      
7521   WARD    SALESMAN  7698  22.02.1981 08:00:00  1250  500   30      
7654   MARTIN  SALESMAN  7698  28.09.1981 07:00:00  1250  1400  30      
7698   BLAKE   MANAGER   7839  01.05.1981 07:00:00  2850        30      
7844   TURNER  SALESMAN  7698  08.09.1981 07:00:00  1500  0     30      
7900   JAMES   CLERK     7698  03.12.1981 08:00:00  950         30    

6 rows selected.

3. We are only interested in rows with a SAL value between 2000 and 4000, but not in SALESMEN.

select 
    j."EMPNO", 
    j."ENAME", 
    j."JOB", 
    j."MGR", 
    to_date( j."HIREDATE", 'YYYY-MM-DD"T"HH  :MI:SS"Z"' ) as "HIREDATE", 
    j."SAL", 
    j."COMM", 
    j."DEPTNO"
from json_table(
    apex_web_service.make_rest_request(
        'http://{server}:{port}/ords/scott/emp/?q=' ||
            utl_url.escape( '
                {"$and": [ 
                    { "sal": { "$between": [ 2000, 4000 ] } }, 
                    { "job": { "$ne": "SALESMAN" } } 
                ] }' ),
        'GET' ),
    '$.items[*]'
    columns (
        "EMPNO"      NUMBER          path '$.empno',
        "ENAME"      VARCHAR2(4000)  path '$.ename',
        "JOB"        VARCHAR2(4000)  path '$.job',
        "MGR"        NUMBER          path '$.mgr',
        "HIREDATE"   VARCHAR2(4000)  path '$.hiredate',
        "SAL"        NUMBER          path '$.sal',
        "COMM"       NUMBER          path '$.comm',
        "DEPTNO"     NUMBER          path '$.deptno'
    )
) j
/

EMPNO  ENAME   JOB       MGR   HIREDATE             SAL   COMM  DEPTNO  
------ ------- --------- ----- -------------------- ----- ----- ------
7499   ALLEN   SALESMAN  7698  20.02.1981 08:00:00  1600  300   30      
7521   WARD    SALESMAN  7698  22.02.1981 08:00:00  1250  500   30      
7654   MARTIN  SALESMAN  7698  28.09.1981 07:00:00  1250  1400  30      
7698   BLAKE   MANAGER   7839  01.05.1981 07:00:00  2850        30      
7844   TURNER  SALESMAN  7698  08.09.1981 07:00:00  1500  0     30      
7900   JAMES   CLERK     7698  03.12.1981 08:00:00  950         30   

5 rows selected.

The most important thing with these examples is that the filter is passed to the REST service - all filtering happens there and only the rows of interest are passed back to the database. When we executed the filter by amending the SQL WHERE clause at the beginning, the opposite was true: All rows have been retrieved from the REST service. But of course, this requires that the REST service supports a filtering feature and provides a syntax.

The packaged application Sample REST Services illustrates how the ORDS filter syntax can be used not only in a SQL query, but in combination with an Application Express classic report.

ORDS filtering syntax in combination with an Application Express Classic Report

ORDS filtering syntax in combination with an Application Express Classic Report

ORDS allows to combine the filtering syntax with the pagination syntax, illustrated in the previous article. So you can have an Application Express report page, displaying data from a REST service, allowing the end user to browse through pages and to execute filters. In all cases, only the data of interest is being fetched from the REST service. Feel free to experiment with this!

back to blogs.oracle.com/apex