docs/db_dev_process/newpart.rst - prjxray - Git at Google


            
            Guide to adding a new device to an existing family

            
            ==================================================


            This documents how to add support for a new device. The running example

            
            is the addition of the xc7a100t device to the Artix-7 family.


            Adding a new device to an existing family is much simpler than adding a

            
            new family, since the building blocks (tiles) are already known. There

            
            are just more or fewer of them, arranged differently. No new fuzzers are

            
            needed. You just need to rerun some fuzzers for the new device to

            
            understand how the tiles are connected to each other and to IOs.


            Note: Since this guide was written, the xc7a100t has become the primary

            
            device in the database, not a secondary device as it was when it was

            
            originally added. Therefore the files currently in the repo don’t match

            
            what is described here. But if you look at the original PRs, they match

            
            what is described in the examples here.


            The main PR from the example is

            
            `#1313 <https://github.com/SymbiFlow/prjxray/pull/1313>`__. Followup

            
            fixes for problems revealed during testing are

            
            `#1334 <https://github.com/SymbiFlow/prjxray/pull/1334>`__ and

            
            `#1336 <https://github.com/SymbiFlow/prjxray/pull/1336>`__.


            Step 0

            
            ~~~~~~


            Fork a copy of https://github.com/SymbiFlow/prjxray on GitHub (go to the

            
            page, click “Fork” button, select your own workspace).


            Clone your fork, and make a new branch, with a name related to the new

            
            device/package:


            ::


               git clone git@github.com:<yourUserID>/prjxray.git

            
               cd prjxray

            
               git checkout -b <new_branch_name>


            Step 1

            
            ~~~~~~


            Follow the Project X-Ray developer setup instructions in the

            
            `documentation <https://symbiflow.readthedocs.io/en/latest/prjxray/docs/db_dev_process/readme.html>`__,

            
            up through Step 7 and choose Option 1 (invoke the

            
            ``./download-latest-db.sh`` script). This script will clone the official

            
            prjxray-db database under ``database/``. The following steps will make

            
            changes under this directory. You may want to put these changes on your

            
            own fork of ``prjxray-db`` for testing. This is explained at the end,

            
            under “Database Updates”.


            Step 2

            
            ~~~~~~


            Add a new settings file. Usually you will start with an existing

            
            settings file and modify it. Assuming you’re in prjxray/,


            ::


               cp settings/<baseline_device>.sh settings/<new_device>.sh

            
               git add settings/<new_device>.sh


            Example:


            ::


               cp settings/artix7_200t.sh settings/artix7_100t.sh

            
               git add settings/artix7_100t.sh


            Update the following values in the new settings file:


            -  ``XRAY_PART`` –

            
               Important: choose a package that is fully bonded (typically the one with

            
               the largest number of pins). If the part that you’re actually interested

            
               in is different (with fewer bonded pins), it will be handled later. In

            
               the running example, the actual part of interest was the xc7a100tcsg324,

            
               since that is on the Arty A7-100T board. But here, the xc7a100tfgg676

            
               part is used; the xc7a100tcsg324 is handled later.


            -  ``XRAY_ROI_TILEGRID`` – modify the bounding boxes to be a tight fit on

            
               your new part.


            -  ``XRAY_IOI3_TILES`` – These tiles need special

            
               handling for an irregularity in Xilinx 7-series FPGAs. See the

            
               `comments <https://github.com/SymbiFlow/prjxray/blob/master/fuzzers/005-tilegrid/generate_full.py#L401>`__

            
               in the 005 fuzzer for more information.


            `This <https://github.com/SymbiFlow/prjxray/blob/master/settings/artix7_100t.sh>`__

            
            is what the new settings file looked like in the example.


            Source this new settings file:


            ::


               source settings/<new_device>.sh


            Step 3

            
            ~~~~~~


            The project needs to know which device is now available and which fabric it

            
            uses. Because some devices share the same fabric, this mapping needs to be

            
            done manually. Edit the device.yaml file for the used family under

            
            settings/<familiy>/device.yaml be adding the device-fabric mapping:


            ::


              # device to fabric mapping

            
              "xc7a200t":

            
                fabric: "xc7a200t"

            
              "xc7a100t":

            
                fabric: "xc7a100t"

            
              "xc7a50t":

            
                fabric: "xc7a50t"

            
              "xc7a35t":

            
                fabric: "xc7a50t"


            Now, generate all device information for the family:


            ::


              make db-prepare-artix7


            Step 4

            
            ~~~~~~


            Edit the top Makefile


            -  Update the Makefile by adding the new device to the `correct

            
               list <https://github.com/tcal-x/prjxray/blob/fbf4dd897d5a1025ebfeb7c51c5077a6b6c9bc47/Makefile#L171>`__,

            
               so that the Makefile generates targets for the new device (used in

            
               Step 4). ``<new_device>`` is the basename of the new settings file

            
               that you just created.


            ::


               <FAMILY>_PARTS=<existing_devices> <new_device>


            -  In our running example, we add ``artix7_100t`` to ``ARTIX_PARTS``:


            ::


               ARTIX_PARTS=artix7_200t artix7_100t


            Step 5

            
            ~~~~~~


            Make sure you’ve sourced your new device settings file (see the end of

            
            step 2) and generated the device information (see the end of set 3). Now it is

            
            time to run some fuzzers to figure out how the tiles on your new device are

            
            connected.


            Make the following target, with ``<new_device>`` as above, and setting

            
            the parallelism factor ``-j<n>`` appropriate for the number of cores

            
            your host has. The make job can benefit from large numbers of cores.


            ::


               make -j<n> MAX_VIVADO_PROCESS=<n> db-part-only-<new_device>


            Again, ``<new_device>`` must match the base name of the new settings

            
            file that was added. For example,


            ::


               make -j32 MAX_VIVADO_PROCESS=32 db-part-only-artix7_100t


            -  It should run fuzzers 000, 001, 005, 072, 073, 074, and 075.


            -  005 will take a long time. Using multiple cores will help.


            -  074 *will fail* the first time, since it hasn’t been told to ignore

            
               certain wires.


               -  After it fails, go to the build directory

            
                  ``cd fuzzers/074-dump_all/build_<XRAY_PART>`` (this is the

            
                  ``XRAY_PART`` from the new settings script; in our example, the

            
                  build directory is

            
                  ``fuzzers/074-dump_all/build_xc7a100tfgg676-1/``).

            
               -  Run

            
                  ``python3 ../analyze_errors.py --output_ignore_list  > new-ignored``

            
               -  Inspect and compare ``new-ignored`` against existing ignored wire

            
                  files in ``../ignored_wires/``.

            
               -  If it looks good, copy it to an appropriately-named file:

            
                  ``cp new-ignored ../ignored_wires/artix7/<XRAY_PART>_ignored_wires.txt``

            
                  (in our example, it is

            
                  ``../ignored_wires/artix7/xc7a100tfgg676-1_ignored_wires.txt``).

            
               -  Add it:

            
                  ``git add ../ignored_wires/artix7/<XRAY_PART>_ignored_wires.txt``


            -  Return to prjxray/ directory, and clean up 074 to prepare for the

            
               rerun: ``make -C fuzzers/074-dump-all clean``


            -  Rerun the top make command,

            
               e.g. ``make -j32 MAX_VIVADO_PROCESS=32 db-part-only-artix7_100t``


            Step 6

            
            ~~~~~~


            The next task is handling the extra parts – those not fully bonded out.

            
            These are usually the parts you actually have on the boards you buy.


            After the fabric data is generated with step 5, an further target can generate

            
            all extra parts for the device.


            ::


               make -j<n> MAX_VIVADO_PROCESS=<n> db-roi-only-<new_device>


            Step 7

            
            ~~~~~~


            Do a spot check.


            -  Check that there are new part directories in the database under the family subdirectory, for example:


            ::


               $ ll database/artix7/xc7a*

            
               xc7a35tftg256-1:

            
               total 48

            
               -rw-rw-r-- 1 daniel daniel  8234 Jan  9 13:01 package_pins.csv

            
               -rw-rw-r-- 1 daniel daniel 18816 Jan  9 13:01 part.json

            
               -rw-rw-r-- 1 daniel daniel 13099 Jan  9 13:01 part.yaml


               xc7a50t:

            
               total 15480

            
               -rw-rw-r-- 1 daniel daniel  695523 Jan  9 12:53 node_wires.json

            
               -rw-rw-r-- 1 daniel daniel 8587682 Jan  9 12:53 tileconn.json

            
               -rw-rw-r-- 1 daniel daniel 6562851 Jan  9 10:31 tilegrid.json


               xc7a50tfgg484-1:

            
               total 52

            
               -rw-rw-r-- 1 daniel daniel 13056 Jan  9 09:54 package_pins.csv

            
               -rw-rw-r-- 1 daniel daniel 18840 Jan  9 09:58 part.json

            
               -rw-rw-r-- 1 daniel daniel 13099 Jan  9 09:58 part.yaml


            Note: These changes/additions under ``database/`` do *not* get checked

            
            in. They are in the ``prjxray-db`` repo. This spot check is to make sure

            
            that your changes in ``prjxray`` will do the right thing when the

            
            official database is fully rebuilt. See “Database Updates” below for

            
            more information.


            Step 8

            
            ~~~~~~


            Assuming everything looks good, commit to your ``prjxray`` fork/branch.

            
            You should have a new file under settings/, a new ignored_wires file,

            
            and a modified Makefile (see the `initial

            
            PR <https://github.com/SymbiFlow/prjxray/pull/1313/files?file-filters%5B%5D=>`__

            
            of the example for reference).


            ::


               git add Makefile settings/artix7_100t.sh

            
               git status

            
               git commit --signoff


            Step 9

            
            ~~~~~~


            Push to GitHub:


            ::


               git push origin <new_branch_name>


            Then make a pull request. Navigate to the GitHub page for your

            
            ``prjxray`` fork/branch, and click the “New pull request” button.

            
            Making the pull request will kick off continuous integration tests.

            
            Watch the results and fix any issues.


            Database Updates

            
            ~~~~~~~~~~~~~~~~


            The process above (steps 4 and 5) will create some new files and modify

            
            some existing files under database/, which is a different repo,

            
            ``prjxray-db``.


            To test these changes before the next official prjxray-db gets built

            
            (and even before your PR on prjxray is merged), you can put these

            
            changes on your own fork of prjxray-db, and then test them in the

            
            context of

            
            `symbiflow-arch-defs <https://github.com/SymbiFlow/symbiflow-arch-defs>`__.


            To put the db updates on your own fork, create your fork of

            
            https://github.com/SymbiFlow/prjxray-db if you haven’t already. Then

            
            follow one of the approaches suggested in the checked solution of this

            
            StackOverflow

            
            `post <https://stackoverflow.com/questions/25545613/how-can-i-push-to-my-fork-from-a-clone-of-the-original-repo>`__.


            You are NEVER going to send a pull request on `prjxray-db`. The database is always rebuilt

            
            from scratch. After your changes on prjxray are merged, they will

            
            reflected in the next prjxray-db rebuild. The changes submitted to your

            
            prjxray-db fork are only for your own testing.


            To use your new repo/branch under

            
            symbiflow-arch-defs/third_party/prjxray-db/, you will need to change the

            
            submodule reference to point to your fork/branch of ``prjxray-db``.
	Guide to adding a new device to an existing family
	==================================================

	This documents how to add support for a new device. The running example
	is the addition of the xc7a100t device to the Artix-7 family.

	Adding a new device to an existing family is much simpler than adding a
	new family, since the building blocks (tiles) are already known. There
	are just more or fewer of them, arranged differently. No new fuzzers are
	needed. You just need to rerun some fuzzers for the new device to
	understand how the tiles are connected to each other and to IOs.

	Note: Since this guide was written, the xc7a100t has become the primary
	device in the database, not a secondary device as it was when it was
	originally added. Therefore the files currently in the repo don’t match
	what is described here. But if you look at the original PRs, they match
	what is described in the examples here.

	The main PR from the example is
	`#1313 <https://github.com/SymbiFlow/prjxray/pull/1313>`__. Followup
	fixes for problems revealed during testing are
	`#1334 <https://github.com/SymbiFlow/prjxray/pull/1334>`__ and
	`#1336 <https://github.com/SymbiFlow/prjxray/pull/1336>`__.

	Step 0
	~~~~~~

	Fork a copy of https://github.com/SymbiFlow/prjxray on GitHub (go to the
	page, click “Fork” button, select your own workspace).

	Clone your fork, and make a new branch, with a name related to the new
	device/package:

	::

	git clone git@github.com:<yourUserID>/prjxray.git
	cd prjxray
	git checkout -b <new_branch_name>

	Step 1
	~~~~~~

	Follow the Project X-Ray developer setup instructions in the
	`documentation <https://symbiflow.readthedocs.io/en/latest/prjxray/docs/db_dev_process/readme.html>`__,
	up through Step 7 and choose Option 1 (invoke the
	``./download-latest-db.sh`` script). This script will clone the official
	prjxray-db database under ``database/``. The following steps will make
	changes under this directory. You may want to put these changes on your
	own fork of ``prjxray-db`` for testing. This is explained at the end,
	under “Database Updates”.

	Step 2
	~~~~~~

	Add a new settings file. Usually you will start with an existing
	settings file and modify it. Assuming you’re in prjxray/,

	::

	cp settings/<baseline_device>.sh settings/<new_device>.sh
	git add settings/<new_device>.sh

	Example:

	::

	cp settings/artix7_200t.sh settings/artix7_100t.sh
	git add settings/artix7_100t.sh

	Update the following values in the new settings file:

	- ``XRAY_PART`` –
	Important: choose a package that is fully bonded (typically the one with
	the largest number of pins). If the part that you’re actually interested
	in is different (with fewer bonded pins), it will be handled later. In
	the running example, the actual part of interest was the xc7a100tcsg324,
	since that is on the Arty A7-100T board. But here, the xc7a100tfgg676
	part is used; the xc7a100tcsg324 is handled later.

	- ``XRAY_ROI_TILEGRID`` – modify the bounding boxes to be a tight fit on
	your new part.

	- ``XRAY_IOI3_TILES`` – These tiles need special
	handling for an irregularity in Xilinx 7-series FPGAs. See the
	`comments <https://github.com/SymbiFlow/prjxray/blob/master/fuzzers/005-tilegrid/generate_full.py#L401>`__
	in the 005 fuzzer for more information.

	`This <https://github.com/SymbiFlow/prjxray/blob/master/settings/artix7_100t.sh>`__
	is what the new settings file looked like in the example.

	Source this new settings file:

	::

	source settings/<new_device>.sh

	Step 3
	~~~~~~

	The project needs to know which device is now available and which fabric it
	uses. Because some devices share the same fabric, this mapping needs to be
	done manually. Edit the device.yaml file for the used family under
	settings/<familiy>/device.yaml be adding the device-fabric mapping:

	::

	# device to fabric mapping
	"xc7a200t":
	fabric: "xc7a200t"
	"xc7a100t":
	fabric: "xc7a100t"
	"xc7a50t":
	fabric: "xc7a50t"
	"xc7a35t":
	fabric: "xc7a50t"

	Now, generate all device information for the family:

	::

	make db-prepare-artix7

	Step 4
	~~~~~~

	Edit the top Makefile

	- Update the Makefile by adding the new device to the `correct
	list <https://github.com/tcal-x/prjxray/blob/fbf4dd897d5a1025ebfeb7c51c5077a6b6c9bc47/Makefile#L171>`__,
	so that the Makefile generates targets for the new device (used in
	Step 4). ``<new_device>`` is the basename of the new settings file
	that you just created.

	::

	<FAMILY>_PARTS=<existing_devices> <new_device>

	- In our running example, we add ``artix7_100t`` to ``ARTIX_PARTS``:

	::

	ARTIX_PARTS=artix7_200t artix7_100t

	Step 5
	~~~~~~

	Make sure you’ve sourced your new device settings file (see the end of
	step 2) and generated the device information (see the end of set 3). Now it is
	time to run some fuzzers to figure out how the tiles on your new device are
	connected.

	Make the following target, with ``<new_device>`` as above, and setting
	the parallelism factor ``-j<n>`` appropriate for the number of cores
	your host has. The make job can benefit from large numbers of cores.

	::

	make -j<n> MAX_VIVADO_PROCESS=<n> db-part-only-<new_device>

	Again, ``<new_device>`` must match the base name of the new settings
	file that was added. For example,

	::

	make -j32 MAX_VIVADO_PROCESS=32 db-part-only-artix7_100t

	- It should run fuzzers 000, 001, 005, 072, 073, 074, and 075.

	- 005 will take a long time. Using multiple cores will help.

	- 074 will fail the first time, since it hasn’t been told to ignore
	certain wires.

	- After it fails, go to the build directory
	``cd fuzzers/074-dump_all/build_<XRAY_PART>`` (this is the
	``XRAY_PART`` from the new settings script; in our example, the
	build directory is
	``fuzzers/074-dump_all/build_xc7a100tfgg676-1/``).
	- Run
	``python3 ../analyze_errors.py --output_ignore_list > new-ignored``
	- Inspect and compare ``new-ignored`` against existing ignored wire
	files in ``../ignored_wires/``.
	- If it looks good, copy it to an appropriately-named file:
	``cp new-ignored ../ignored_wires/artix7/<XRAY_PART>_ignored_wires.txt``
	(in our example, it is
	``../ignored_wires/artix7/xc7a100tfgg676-1_ignored_wires.txt``).
	- Add it:
	``git add ../ignored_wires/artix7/<XRAY_PART>_ignored_wires.txt``

	- Return to prjxray/ directory, and clean up 074 to prepare for the
	rerun: ``make -C fuzzers/074-dump-all clean``

	- Rerun the top make command,
	e.g. ``make -j32 MAX_VIVADO_PROCESS=32 db-part-only-artix7_100t``

	Step 6
	~~~~~~

	The next task is handling the extra parts – those not fully bonded out.
	These are usually the parts you actually have on the boards you buy.

	After the fabric data is generated with step 5, an further target can generate
	all extra parts for the device.

	::

	make -j<n> MAX_VIVADO_PROCESS=<n> db-roi-only-<new_device>

	Step 7
	~~~~~~

	Do a spot check.

	- Check that there are new part directories in the database under the family subdirectory, for example:

	::

	$ ll database/artix7/xc7a*
	xc7a35tftg256-1:
	total 48
	-rw-rw-r-- 1 daniel daniel 8234 Jan 9 13:01 package_pins.csv
	-rw-rw-r-- 1 daniel daniel 18816 Jan 9 13:01 part.json
	-rw-rw-r-- 1 daniel daniel 13099 Jan 9 13:01 part.yaml

	xc7a50t:
	total 15480
	-rw-rw-r-- 1 daniel daniel 695523 Jan 9 12:53 node_wires.json
	-rw-rw-r-- 1 daniel daniel 8587682 Jan 9 12:53 tileconn.json
	-rw-rw-r-- 1 daniel daniel 6562851 Jan 9 10:31 tilegrid.json

	xc7a50tfgg484-1:
	total 52
	-rw-rw-r-- 1 daniel daniel 13056 Jan 9 09:54 package_pins.csv
	-rw-rw-r-- 1 daniel daniel 18840 Jan 9 09:58 part.json
	-rw-rw-r-- 1 daniel daniel 13099 Jan 9 09:58 part.yaml

	Note: These changes/additions under ``database/`` do not get checked
	in. They are in the ``prjxray-db`` repo. This spot check is to make sure
	that your changes in ``prjxray`` will do the right thing when the
	official database is fully rebuilt. See “Database Updates” below for
	more information.

	Step 8
	~~~~~~

	Assuming everything looks good, commit to your ``prjxray`` fork/branch.
	You should have a new file under settings/, a new ignored_wires file,
	and a modified Makefile (see the `initial
	PR <https://github.com/SymbiFlow/prjxray/pull/1313/files?file-filters%5B%5D=>`__
	of the example for reference).

	::

	git add Makefile settings/artix7_100t.sh
	git status
	git commit --signoff

	Step 9
	~~~~~~

	Push to GitHub:

	::

	git push origin <new_branch_name>

	Then make a pull request. Navigate to the GitHub page for your
	``prjxray`` fork/branch, and click the “New pull request” button.
	Making the pull request will kick off continuous integration tests.
	Watch the results and fix any issues.

	Database Updates
	~~~~~~~~~~~~~~~~

	The process above (steps 4 and 5) will create some new files and modify
	some existing files under database/, which is a different repo,
	``prjxray-db``.

	To test these changes before the next official prjxray-db gets built
	(and even before your PR on prjxray is merged), you can put these
	changes on your own fork of prjxray-db, and then test them in the
	context of
	`symbiflow-arch-defs <https://github.com/SymbiFlow/symbiflow-arch-defs>`__.

	To put the db updates on your own fork, create your fork of
	https://github.com/SymbiFlow/prjxray-db if you haven’t already. Then
	follow one of the approaches suggested in the checked solution of this
	StackOverflow
	`post <https://stackoverflow.com/questions/25545613/how-can-i-push-to-my-fork-from-a-clone-of-the-original-repo>`__.

	You are NEVER going to send a pull request on `prjxray-db`. The database is always rebuilt
	from scratch. After your changes on prjxray are merged, they will
	reflected in the next prjxray-db rebuild. The changes submitted to your
	prjxray-db fork are only for your own testing.

	To use your new repo/branch under
	symbiflow-arch-defs/third_party/prjxray-db/, you will need to change the
	submodule reference to point to your fork/branch of ``prjxray-db``.