Merge branch 'INPLATFORM-1827' into 'main'

INPLATFORM-1827: Migrate Apigee docs and api standard

See merge request !1

.gitignore

+.idea

README.md

 # API Publisher Documentation
 
+This repository contains documentation for API Publishers who create institutional APIs for the UW Developer Portal (https://developer.wisc.edu).
 
+[[_TOC_]]
 
-## Getting started
+## UW-Madison API Standards
 
-To make it easy for you to get started with GitLab, here's a list of recommended next steps.
+All APIs must meet a set of requirements in order to be published on the UW Developer Portal.
+See [this document](./api-standards.md) for details.
 
-Already a pro? Just edit this README.md and make it your own. Want to make it easy? [Use the template at the bottom](#editing-this-readme)!
+## Apigee Documentation
 
-## Add your files
+Google Apigee is an API Management platform used for managing access to institutional APIs.
+See [the documentation](./apigee.md) for details and steps to get access.
 
-- [ ] [Create](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#create-a-file) or [upload](https://docs.gitlab.com/ee/user/project/repository/web_editor.html#upload-a-file) files
-- [ ] [Add files using the command line](https://docs.gitlab.com/ee/gitlab-basics/add-file.html#add-a-file-using-the-command-line) or push an existing Git repository with the following command:
+## Feedback and questions
 
-```
-cd existing_repo
-git remote add origin https://git.doit.wisc.edu/interop/external-docs/api-publisher-documentation.git
-git branch -M main
-git push -uf origin main
-```
-
-## Integrate with your tools
-
-- [ ] [Set up project integrations](https://git.doit.wisc.edu/interop/external-docs/api-publisher-documentation/-/settings/integrations)
-
-## Collaborate with your team
-
-- [ ] [Invite team members and collaborators](https://docs.gitlab.com/ee/user/project/members/)
-- [ ] [Create a new merge request](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html)
-- [ ] [Automatically close issues from merge requests](https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#closing-issues-automatically)
-- [ ] [Enable merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/approvals/)
-- [ ] [Set auto-merge](https://docs.gitlab.com/ee/user/project/merge_requests/merge_when_pipeline_succeeds.html)
-
-## Test and Deploy
-
-Use the built-in continuous integration in GitLab.
-
-- [ ] [Get started with GitLab CI/CD](https://docs.gitlab.com/ee/ci/quick_start/index.html)
-- [ ] [Analyze your code for known vulnerabilities with Static Application Security Testing (SAST)](https://docs.gitlab.com/ee/user/application_security/sast/)
-- [ ] [Deploy to Kubernetes, Amazon EC2, or Amazon ECS using Auto Deploy](https://docs.gitlab.com/ee/topics/autodevops/requirements.html)
-- [ ] [Use pull-based deployments for improved Kubernetes management](https://docs.gitlab.com/ee/user/clusters/agent/)
-- [ ] [Set up protected environments](https://docs.gitlab.com/ee/ci/environments/protected_environments.html)
-
-***
-
-# Editing this README
-
-When you're ready to make this README your own, just edit this file and use the handy template below (or feel free to structure it however you want - this is just a starting point!). Thanks to [makeareadme.com](https://www.makeareadme.com/) for this template.
-
-## Suggestions for a good README
-
-Every project is different, so consider which of these sections apply to yours. The sections used in the template are suggestions for most open source projects. Also keep in mind that while a README can be too long and detailed, too long is better than too short. If you think your README is too long, consider utilizing another form of documentation rather than cutting out information.
-
-## Name
-Choose a self-explaining name for your project.
-
-## Description
-Let people know what your project can do specifically. Provide context and add a link to any reference visitors might be unfamiliar with. A list of Features or a Background subsection can also be added here. If there are alternatives to your project, this is a good place to list differentiating factors.
-
-## Badges
-On some READMEs, you may see small images that convey metadata, such as whether or not all the tests are passing for the project. You can use Shields to add some to your README. Many services also have instructions for adding a badge.
-
-## Visuals
-Depending on what you are making, it can be a good idea to include screenshots or even a video (you'll frequently see GIFs rather than actual videos). Tools like ttygif can help, but check out Asciinema for a more sophisticated method.
-
-## Installation
-Within a particular ecosystem, there may be a common way of installing things, such as using Yarn, NuGet, or Homebrew. However, consider the possibility that whoever is reading your README is a novice and would like more guidance. Listing specific steps helps remove ambiguity and gets people to using your project as quickly as possible. If it only runs in a specific context like a particular programming language version or operating system or has dependencies that have to be installed manually, also add a Requirements subsection.
-
-## Usage
-Use examples liberally, and show the expected output if you can. It's helpful to have inline the smallest example of usage that you can demonstrate, while providing links to more sophisticated examples if they are too long to reasonably include in the README.
-
-## Support
-Tell people where they can go to for help. It can be any combination of an issue tracker, a chat room, an email address, etc.
-
-## Roadmap
-If you have ideas for releases in the future, it is a good idea to list them in the README.
-
-## Contributing
-State if you are open to contributions and what your requirements are for accepting them.
-
-For people who want to make changes to your project, it's helpful to have some documentation on how to get started. Perhaps there is a script that they should run or some environment variables that they need to set. Make these steps explicit. These instructions could also be useful to your future self.
-
-You can also document commands to lint the code or run tests. These steps help to ensure high code quality and reduce the likelihood that the changes inadvertently break something. Having instructions for running tests is especially helpful if it requires external setup, such as starting a Selenium server for testing in a browser.
-
-## Authors and acknowledgment
-Show your appreciation to those who have contributed to the project.
-
-## License
-For open source projects, say how it is licensed.
-
-## Project status
-If you have run out of energy or time for your project, put a note at the top of the README saying that development has slowed down or stopped completely. Someone may choose to fork your project or volunteer to step in as a maintainer or owner, allowing your project to keep going. You can also make an explicit request for maintainers.
+Changes to the documentation in this repository are welcome.
+Please create a merge request with your suggested changes.
+For any other feedback or questions, please contact the DoIT Enterprise Integration API Team: api@doit.wisc.edu

api-standards.md

+UW-Madison API Standards
+========================
+
+This document describes standards each UW API must meet to be considered for inclusion in the Developer Portal ([https://developer.wisc.edu](https://developer.wisc.edu))
+
+These standards are maintained by the Enterprise Integration API Team with consultation and support from other API producers and stakeholders.
+Please contact [api@doit.wisc.edu](mailto:api@doit.wisc.edu) with questions and feedback.
+
+Motivation
+----------
+
+*   UW-Madison has an overarching data documentation and integration standards, both of which leave room for API-specific standards to be curated by DoIT.
+
+    *   [Institutional Data Documentation Standard](https://uwmadison.app.box.com/s/xvi4mqicuiiezoqi6njywi3cow63z65h)
+        
+    *   [Institutional Data Integration Standard](https://uwmadison.app.box.com/s/wla5zksaeab8wk299uo69qqjzfevzg7c)
+        
+    *   See also: [Policies, Procedures, and Standards](https://data.wisc.edu/data-governance/policies-procedures/)
+        
+*   Each API in the Developer Portal should conform to a minimum standard. It is important that we provide a consistent user experience, data, security, and authorization practices across UW APIs.
+
+*   APIs and their documentation should be easy to find, understand, and maintain. Lowering the barrier to using APIs through good, consistent documentation will promote adoption of APIs, reduce support burdens on API producers, and will increase likelihood of integrator success.
+    
+
+API Standard
+------------
+
+*   If an integrator has a reasonable expectation of being approved for an API that they need for a valid business reason or integration, then that API is likely to be a good candidate for inclusion in the Developer Portal.
+
+*   If an API is tailored to a specific use case or expresses data unique to a particular consuming system, it is likely not appropriate to include in the Developer Portal.
+
+*   If an API’s purpose is narrow in scope, but might provide value to many consumers, irrespective of adoption, it may be considered broadly useful.
+
+*   **Abstracted:** APIs that express institutional data as defined by the [Data Governance Council](https://data.wisc.edu/data-governance/data-domains-trustees-and-stewards/) should present appropriate domain data models and not vendor-specific terminology.
+    
+*   **Naming:** The name of an API presented in the Developer Portal should also reflect abstraction, avoiding vendor-specific naming where possible. If acronyms are necessary in the API name, they should be spelled out and explained in the documentation.
+    
+*   **Mock and Test:** API Producers are encouraged to provide testing functionality with sample data that do not require an approval process. APIs that present sample data should be prefaced with the word “Mock”. Test versions of APIs with test data may also be presented in the Developer Portal, may require authorization, and may be separate than Mock APIs.
+
+*   **Risk management:** Each API must comply with UW-Madison’s Cybersecurity risk management framework as [outlined here](https://it.wisc.edu/about/division-of-information-technology/enterprise-information-security-services/office-of-cybersecurity/risk-management-and-compliance-rmc/).In practice, this usually means undergoing a Cybersecurity Consultative Review (CCR) to assess, manage, and mitigate risks.
+
+*   **Well documented:** Each API must have up-to-date documentation that enables API consumers to easily get started and use the API. This documentation must conform to the requirements outlined below.
+
+*   **Coordination of Boundaries:** APIs that express data across multiple domain boundaries should be coordinated, data should be expressed in alignment with institutional models, and that data should be sourced from an appropriate system of record or reference (see example below).
+
+### Example of Coordination of Boundaries
+
+If a Curricular and Academic API needs to present Person data, then API Producer teams should coordinate whether the Curricular and Academic API will reference the Person API in its responses (i.e.hand off) or if data is presented directly using common data models.
+
+If the Curricular and Academic API expresses Person data directly, it might use a near real-time mechanism like webhooks to update that data from the Person API to allow data to remain synchronized in both APIs. The data should also be presented using a common data model as defined by each specification.
+
+Similarly, if the Curricular and Academic API expresses data from the Person API, both should have similar classification and authorization processes, as defined by data governance. Data restricted from a given application using the Person API should not be accessible through the Curricular and Academic API.
+
+Documentation Standard
+----------------------
+
+### Presentation
+
+*   **Cross-linking:** Documentation should either be housed within or linked from within the Developer Portal and accessible to the same population of integrators without additional authorization.
+
+*   **Visuals:** APIs should not have logos or other marketing-style material since it is challenging to synchronize style across disparate API producer teams.
+
+### Required
+
+The topics below are required documentation for each API, and producers must actively maintain this documentation for an API to be included in the Developer Portal.
+
+*   **Getting Access**
+
+    *   Authorization workflow, timeline, and expectations
+
+    *   Terms of use
+
+*   **Specification**
+
+    *   OpenAPI/Swagger for RESTful APIs
+
+        *   Documentation must be linked from the info.description section so it can be found from within other tools like Postman and IDEs
+
+    *   Data dictionary (or external link if applicable)
+
+    *   Data classification(s)
+
+*   **Integrator Guidelines**
+
+    *   Audience for the API
+
+    *   Example use cases for the API
+
+    *   Error management practices
+
+        *   Error behaviors and definitions
+
+        *   Retry and backoff guidance
+
+    *   How to perform or simulate
+
+        *   End-to-end testing
+
+        *   Load testing
+
+*   **Operations**
+
+    *   Service level agreements (SLA) or objectives (SLO) and related metrics
+
+    *   Versioning strategy
+
+    *   Rate limits and throttling
+
+    *   Release notes
+
+    *   How to obtain support
+
+
+### Recommended
+
+The API Team also recommends documenting the supporting topics below, but these are not strictly required for inclusion in the Developer Portal.
+
+*   **Tutorials**
+
+    *   Using the API with tools like Postman
+
+    *   Postman collections or configurations for other tooling
+
+    *   How to use JSON:API includes, filters, pagination, webhooks, etc.
+
+    *   Transcript or text version of video tutorials
+
+*   **Developer Support**
+
+    *   Templates or code snippets
+
+    *   Clients (where applicable and actively maintained)
+
+    *   Community supported or external resources
+
+    *   Troubleshooting guides
+
+    *   Related APIs and how to work with data across multiple APIs
\ No newline at end of file

apigee.md

+# Apigee
+
+"Apigee is a platform for developing and managing APIs. By fronting services with a proxy layer, Apigee provides an abstraction or facade for your backend service APIs and provides security, rate limiting, quotas, analytics, and more."
+
+https://cloud.google.com/apigee/docs/api-platform/get-started/what-apigee
+
+Apigee is the API Management platform for managing institutional APIs at UW-Madison.
+
+[[_TOC_]]
+
+## Apigee Training
+
+We currently have access to a limited number of licenses for Apigee training through [Coursera](https://www.coursera.org). This training, which is a mix of lectures and hands-on labs, provides an introduction to the API publishing capabilities available within the Apigee platform.
+
+To request to use a license [contact the API Team](mailto:api@doit.wisc.edu). Upon review of your request and if a license is available, you will receive an email from Coursera with a link to join the Apigee training program for UW-Madison.
+
+## Getting Access to Publish APIs
+
+Apigee is meant for teams that want to publish APIs for others to use.
+When others browse APIs and get access to use them, they will do so through a Developer Portal instead of interacting with Apigee directly.
+
+_The API Team is currently onboarding publishers to Apigee on a limited basis._
+
+This is due to the low level of isolation that exists within Apigee.  While Apigee provides granular control over role-based permissions, it does not provide a means for separating access so that multiple teams can exist in the same Apigee organization without being able to access the other's APIs and related resources.
+
+Publishers that choose to use Apigee to publish and manage their API should be aware of these limitations:
+
+- All API publishers will have the same access to the development, pre-production, and production Apigee organizations.
+- All API publishers will have access to view and edit all API proxies, API products, and API documentation published to the developer portal.
+- All API publishers will be able to view and edit developer applications subscribed to their APIs or anyone else's APIs. _This includes viewing OAuth2 keys and secrets associated with a developer application._
+- All API publishers will be able to view and modify information used to connect to backend services (the actual API that the Apigee API proxy is sending traffic to) _including credentials and secrets._
+
+If these limitations *do not* meet the needs of your use-case, please [contact the API Team](mailto:api@doit.wisc.edu) to give us specific feedback and to stay updated when Apigee can better handle multiple teams coexisting in the same Apigee organization.
+
+If these limitations *do* meet the needs of your use case, please follow the instructions below to request access to Apigee:
+
+1. Contact the API Team (api@doit.wisc.edu) and include your name, the team/department you're representing, and an overview of your use-case for using Apigee. Someone will be in contact with you between 1-3 business days to gather more information or schedule a call to discuss further.
+2. After confirming the API Team is able to onboard your team to Apigee, [create a Google Group](https://kb.wisc.edu/googleapps/page.php?id=98121) that contains the team members that you would like to allow access to Apigee. Send the name of the group (e.g. my-team@g-groups.wisc.edu) to the API Team. This group will be controlled by your team and is used to provide or remove your team's access to Apigee. Note, your team may only consist of yourself, but it is still beneficial to have access controlled through a group in case API ownership needs to be transferred in the future.
+
+## Login to Apigee
+
+Login to Apigee here: https://apigee.google.com/
+
+Apigee uses Google and NetID login for authentication. Make sure you are logged into Google using your NetID.
+
+If prompted, make sure you are using the project `doit-ipt-apigee-dev-808d`
+
+## Onboarding a New API
+
+1. Select `doit` environment to deploy an API proxy
+    * UW-Madison currently has a limited licensing of 3 Apigee organizations (development, pre-prod, prod), and 15 Environments across all Apigee organizations. If new environments or any changes are needed, please contact the API Team (api@doit.wisc.edu) to create/edit environments.
+1. [Create a reverse proxy for your API](https://cloud.google.com/apigee/docs/api-platform/fundamentals/understanding-apis-and-api-proxies)
+    * A reverse proxy can also be created using the VS Code Apigee plugin as described in
+      [Developing API proxies](https://cloud.google.com/apigee/docs/api-platform/local-development/vscode/develop-apiproxies)
+    * If your API is IP restricted, add these IP addresses to your firewall's allow-list for each Apigee organization.
+        * dev: 34.71.118.160
+        * preprod: 34.134.105.239
+        * prod: 35.238.42.232
+
+1. [Publish your API](https://cloud.google.com/apigee/docs/api-platform/publish/publishing-overview).
+
+## Authorizing Calls to Your API (OAuth2)
+
+[OAuth2](https://cloud.google.com/apigee/docs/api-platform/security/oauth/oauth-introduction) is the preferred method for authorizing calls to APIs deployed in Apigee.
+The API Team maintains an Apigee proxy that can be used to get an access token to use for authorizing calls to other APIs.
+API publishers do not need to create their own OAuth2 proxy, but they do need to configure their proxies to check the OAuth2 access token before proceeding with the rest of the request.
+More information on this process can be found in the [OAuthV2 policy documentation](https://cloud.google.com/apigee/docs/api-platform/reference/policies/oauthv2-policy#verifyaccesstoken).
+
+The API specification for the OAuth proxy (i.e. for getting an access token) can be [found on the Developer Portal](https://developer.wisc.edu/docs/oauth/1/overview).
+
+### Limitations
+
+* Access tokens issues by the OAuth proxy can only be used with APIs in the same Apigee organization. For example, a token issued in the dev organization cannot be used in the production org, and vice-versa.
+* The OAuth proxy currently only supports the Client Credentials grant type. If you have a use-case that requires other grant types, contact the API Team (api@doit.wisc.edu) for further discussion.
+
+## Developer Portal
+
+The Developer Portal is where API consumers go to register for and view documentation about APIs.
+Unlike API publishers, consumers do not interact directly with the Apigee console.
+Apigee is a backend implementation detail of the overall experience of discovering, registering for, and using an API.
+
+Each Apigee organization has its own Developer Portal.
+Non-production Developer Portals should only be used for API teams to test their own APIs, as well as for getting feedback from stakeholders on APIs before they are released in production.
+Otherwise, the [production Developer Portal](https://developer.wisc.edu/) should be the only location where API consumers discover and register for APIs.
+
+![API development vs. production environments](./static/dev-prod-domains.png)
+
+If your API requires additional environments to be able to satisfy the use-cases of your API (i.e. in addition to the mock and non-mock environments in the production Apigee organization), please [contact the API Team](mailto:api@doit.wisc.edu) so that we can discuss your use-case.
+Separate Apigee organizations, and therefore separate Developer Portals, should not be used as a substitute for additional environments.
+
+### Allowing Developer Portal Access to Your API
+
+In order for consumers to call your API from the developer portal, you need enable the developer portal CORS policy:
+
+* Create a new [Flow Callout policy](https://cloud.google.com/apigee/docs/api-platform/reference/policies/flow-callout-policy) in your API proxy that uses the `SF_CORS_DEVPORTAL` shared flow.
+* Add this policy to the request pre-flow of your API proxy.
+
+## Using GitLab Runner to Publish APIs
+
+The API Team can create a Gitlab runner for publishing APIs using Gitlab's CI/CD pipelines. Your Gitlab runner will only
+be able to modify Apigee resources that begin with your team's assigned prefix.  If you are interested in using this in
+your own projects, [contact the API Team](mailto:api@doit.wisc.edu) with the following information to have this runner
+enabled for your GitLab workspace.
+
+1. Your preferred team prefix:
+    * 10 characters or fewer
+    * Only lowercase letters, numbers, and hyphens
+    * Starts with an alphabetic character
+    * Ends with an alphabetic or numeric character
+    * For example: "api-team" or "asp-api"
+
+2. A shared secret for your GitLab Runner token:
+    * From the workspace in GitLab where you want to use the runner, go to CI/CD > Runners and click the
+      "Register a group runner" button to access the token.
+    * Put this token in a secure location and send us that location in your email.
+
+## Logging in Apigee
+
+### [API Monitoring](https://cloud.google.com/apigee/docs/api-monitoring)
+
+After you build and launch your APIs on Apigee, you need to ensure that they are available and performing as expected in
+order to maintain uninterrupted service. Apigee's API Monitoring enables you to track your APIs to make sure they are up
+and running correctly. API Monitoring provides near real-time insights into API traffic and performance, to help you
+quickly diagnose and solve issues as they arise.
+
+### [Developer Engagement](https://cloud.google.com/apigee/docs/api-platform/analytics/partner-engagement-dashboard)
+
+The Developer Engagement dashboard tells you which of your registered app developers are generating the most API traffic.
+For each of your developers, you can find out who is generating the most API traffic and the most errors. For example,
+if a particular developer's app is generating a lot of errors relative to other developers, you can pro-actively address
+the problem with that developer.
+
+### Proxy Usage Logs
+
+Detailed information about who requested what data from each proxy is logged in GCP.  These logs are accessed through the
+[Log Explorer](https://cloud.google.com/logging/docs/view/logs-explorer-interface) by searching for the log name
+`apigee-proxy-usage`.  To look at only logs for your proxy, press the `Show Query` toggle and add the filter
+`jsonPayload.proxy={proxy-name}`.
+
+To filter logs based on elapsed time, add the filter `jsonPayload.time_elapsed>value` or `jsonPayload.time_elapsed<value`.
+([Supported comparison operators](https://cloud.google.com/logging/docs/view/logging-query-language#comparison_operators))
+
+![Proxy Usage Logs in GCP](static/Apigee Proxy Usage Logs.png "Proxy Usage Logs in GCP")
\ No newline at end of file